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PREFACE 



It is already a tradition that conferences on operations research are organized 
by the Mathematisches Forschungsinstitut in Oberwolfach/ Germany. 

The mean point of the 1987 conference was to discuss recently developed methods in 
optimization theory derived from various fields of mathematics. On the other hand, 
the practical use of results in operations research is very important. In the last 
few years- essential progress in this direction was made at the International Insti- 
tute for Applied Systems Analysis ( I I ASA) at Laxenburg/Austria. Therefore a three 
days workshop on Advanced Computation Techniques, Parallel Processing and Optimi- 
zation organized by I I ASA and the University of Karlsruhe immediately followed the 
Oberwolfach Conference. 

This volume contains selected papers which have been presented at one of these 
conferences. 1 1 ;i s divided into five sections based on the above topics: 

I. Algorithms and Optimization Methods 

II. Optimization and Parallel Processing 

III. Graph Theory and Scheduling 

IV. Differential Equations and Operator Theory 

V. Applications- 

We would like to thank the director of the Mathematisches Forschungsinstitut 
Oberwolfach Prof. Dr. M. Barner and the International Institute for Applied Systems 
Analysis, particularly Prof. Dr. V. Kaftanov, and also to the director of the 
Computer Center of the University of Karlsruhe Prof. Dr. A. Schreiner for their 
support in organizing these conferences. We also appreciate the excellent coopera- 
tion of Springer Verlag. 

We also thank Dr. P. Recht, Dr. D. Solte and Dr. K. Wieder as well as Mrs. 
Dagmar Hoffmann, Korina Recht and Monika Schoppa for their help during these con- 
ferences. 
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I. Algorithms and Optimization Methods 




QUANTITATIVE STABILITY OP (MIXED-) INTEGER 
LINEAR OPTIMIZATION PROBLEMS 



B. Bank and R. Mandel 

Sektion Mathematik 
Humboldt Universitat Berlin 
Berlin, DDR 



Abstract, 

We treat (mixed-) integer linear optimization problems with variable 
right-hand sides and an integer constraint matrix from a point of 
view of quantitative stability, i. e. we are looking for a (quasi-) 
Lipschitzian characterization of the constraint set-mapping, the 
value f vinction and the optimal set-function. 

Key words: (Mixed-) Integer Programming, Sensitivity, Stability, 

Parametric Programming, Penalty Functions. 



4. Introduction 

This paper deals with a stability analysis of the (mixed-) integer 
linear optimization problem with perturbed right hand sides 

PO>) inf {c T x |x fcG(b) 1 , b 6R m 
where 

G(b) = {xeM(b) | x>|,...,x integer?, 

Df 

M(b) = {x 6R n | Ax = b, x^O? , 

Df 

s an arbitrarily fixed integer with O^s^n. 

We suppose throughout all the text that the assumption 
(GA) A is an (m,n)-integer matrix 
is satisfied. 

The aim of our considerations consists in the presentation of 
quantitative stability results for parametric problems P(b), where 
quantitative stability will be characterized in terms of the quasi- 
Lipschitzian behaviour of the constraint sets, the value function 
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and the optimal set-function. 

R n 

A multifunction T : A — > 2 M C R m ) is said to be quasi- 

Lipschitzian on a subset /^ Q C ^ if there are constant numbers § 
and T (not depending on Ji€ A) such that 

d(r(A), P( A'))^slU- A* II + ? VjL t A'e A 0 , 

where || • || denotes the euclidean norm and 

d(C,D) = inf {6 |ccu f D, D crU^C 1 , 

Df C 

the Hausdorff-distance of the sets C and D (U^C is the usual 
£ -neighbourhood of the set C). 

R* 1 

The constraint set mapping G : B — ► 2 is defined by G(b) on 

B = {b£R m | G(b)j 0 0 . 

Df 

The value function cp of P(b) defined on B, is given by 

cp(b) = inf fc x |x£G(b)? and its optimal set-function ip given 
Df . m 

by the optimal sets 'y(b) = {xeG(b) | c x x = <p(b)J. 

The theorems presented here extend some of the important results 
(saying that the mappings G, cp and ^ are quasi-Lipschitzian 
on B) obtained by BLAIR/ JEROSLOW (1977, 1979, 1982) which are de- 
rived by the authors in relation to the unified approach in their 
monography on (nonlinear) integer parametric programming ( BA1TK/MANDEL 
1987; see also BAMC/BELOUSOV et al. 1986). The constants appearing 
in this paper with respect to the statements on the quasi-Lipschitzian 
behaviour of the three mappings defined above are developed on the 
basis of estimates given there. Another extension of BLAIR* s and 
JEROSLOW* s results one can find in a recent paper of COOK et al. 

(1986) where good estimates for the constants arising in BLAIR/ 
JEROSLOW (1977, 1979, 1982) are shown. 

The class of parametric problems P(b) we treat is of interest in 
its own rights and, similar as in linear programming, important for 
dealing with duality and exact penalty approaches for (mixed-) inte- 
ger linear programming. 

Applying the presented stability results we prove two theorems, 
originally due to BLAIR/ JEROSLOW (1981), on exact penalty functions 
for the both cases of pure- and mixed- integer linear optimization. 

Eor convenience we use the assumption that cp ( 0 ) = 0 which ensures 
that the optimal set 'V(b) / 0 for all b edom G = B; this guaran- 
tees that the optimal set ^o^), b 6B, corresponding to the linear 
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programming relaxation P Q (b) of the problem P(b) is non empty, 
too. 



2. Stability Results 

p 3 

In KUMMER (i977) it is shown that the multifunction C: dom M — ► 2 
defined by the compact convex polyhedra 



C(b) = fxfcM(b) | -e T x>a(b)j 
Df 



M> 



where 



e T = ('I, £R n , 

a(b) = min {(b T ,-b :r ,0 T )(w 1 - v 1 ^ , 

Df -Ui^r 

{(v i ,w 1 ) |i= / I,...,r] - the set of all vertices of 

the polyhedron 

W a f(v,w) | (A T ,-A T ,-I)(v- w) = e, v,w>Oj , 

Df 

I - the (n,n)-identity matrix 



is continuous on dom M and, it holds that 



M(b) = C(b) + V , b 6 dom M 

where V is the common recession cone of all M(b), i. e. 

V = {x€ R n | Ax = 0, x^O j . 

Df 

Thereby continuity is used in^the following sense. 

A multifunction P: A 2^ is said to be upper semicontinuous 
(according to HAUSDORFF) at 6 A if 

Ve>0 36 = 6(£)>0: P(A)CU 6 T( A°) VA€U 6 {A°i 
and, lower 3 emi continuous (according to BERGE) at A 0 € A if 
VQc R n open: Q n P( a 0 ) 4 0 

=£> 36 = 6(Q ) > Os T(A)nQ j* 0 ^AeU^A 0 ] . 

P is said to be continuous at A°e A if it is both upper and 
lower semicontinuous. 

For the compact set C(b) from (4) there is a constant number § 
such that 



II x || ^gllbll 



Vxec(b) , 



( 2 ) 
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which one can prove using CRAMER's rule and HADAMARD's inequality 
(see e. g. BARK/MARDEL 1987) • Then one easily verifies that C, 

defined by (1), is Lipschitzian on dom M, i. e. there is a constant 

g such that 

d(C(b),C(b» )) £ glib - b* || V b,b* € dom M (3) 

which is a version of HOFFMAR's (1952) famous result (see e. g. 
WALKUP/WETS (1969)). A trivial consequence of these results is the 
fact that the constraint set mapping M of the linear programming 
relaxation P Q (b) of the integer problem P(b) shows continuity on 

dom M. Furthermore M is Lipschitzian on dom M with the same 

constant number g appearing in (3)« 

One of our goals is to establish comparable propositions on the con- 
straint set mapping G of the problem P(b) and, in particular, for 
its optimal set function Ap. 

In BARK (1977) the following sets 

r 

K(b) = { x e R n | x = u +v, u 6 C(b) , v = a.v^, 

i=1 1 

a i 6 [0, 1] , i= 1, . . . ,r, 

x,|,...,x g integer], (4) 

V s = {x£V |x^,^.,x s integer ] 

are defined where (v^,...,v r ] is a system of (mixed-) integer 
generators of the polyhedral cone V (such systems exists by (GA) 
and, then K(b) 4 0 on B) and C(b) is taken from (1). The multi- 

Rn 

function K: B — *2 is compact-valued, upper semi continuous on B 

and, it holds that 

G(b) = K(b) + V a V b 6B . (5) 

s 

((GA) is an essential assumption for the upper semicontinuity of K. ) 

Consequences of these facts are that G is upper semi continuous on 

B and that the set B is closed. 

r ± 

Setting for e. g. T = 5" ||v || and applying the estimate (2) one 

-i — A 

obtains that 

II ac || 4 glib || + X V x €K(b) . (6) 

* 

In order to construct a compact-valued multifunction K*: B — >2 
having the property (5) and which additionally shows quasi-Idpschitzian 
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behaviour on B we make use of the uniform distribution of the set 
G(b) in the polyhedron M(b). We say that the set 

T a =f | . . . .Xg integer] is uniformly distributed in TCR n 

if T 4 0 and 6 = sup inf ||x-y | [*<*, i. e. there is a 6 such that 

xeT yel 

s 

U 6 (x) ft T g £ 0 VxfcT 
(see e. g. BELOUSOV (1977)). 

If (GA) is satisfied, then one easily shows that G(b) 4 0 is uni- 
formly distributed in M(b); in Theorem 1. it will also be shown that 
the corresponding 6(b) is bounded from above on the set B. 

Theorem 1, 

Let (GA) be satisfied. Then 

(i) G is quasi-Lipschitzian on B; 

(ii) there is a constant number 6>0 (not depending on b) such 
that 

6 > sup inf || x-y II V b 6B; 

xeM(b) yeG(b) 

(iii) there is a compact-valued, quasi-Lipschitzian multifunction 
K*: B — > 2^ such that 

G(b) = K*(b) + V Vb6B. 

s 



JProof : (i) We define the sets 

G(q) = {(y,u)eR 2n |Ay-Au = q, y^O, u^-0, 

Df 

y/l»...,y s , u u g integer] 

and ( 7 ) 

V 2S = {(y,u) 6R 2n I Ay - Au = 0, y £0, u 5 0, 

y/|....»yg, u u g integer]. 

Then, defining with respect to G(q) compact sets K(q) according 
to (4) and using constant numbers § and t such that (6) holds 
for K(q) one has 

G(q) = K(q) + V 2g V q fe dom G 

and ( 8 ) 

l|(y,u)|| ^g||q||+r V (y,u) 6 K(q) . 

For arbitrarily fixed points b,b*6B and xfcG(b) we construct a 
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point x*6G(b*) such that 

|{ x - x *|| ^ 2 g lib - b* II + 2 v 



( 9 ) 



Let xeG(b*) be fixed. Then, (x, x) £ G(b-b*) and, by (8), there axe 
points (y,u)6K(b-b*) and (y,u)eV 2s such that 

(x, x) = (y,u) + (y,u) . 

The point x* = u + y belongs to G(b*) and satisfies (9)* Now the 
Df 

validity of (i) is an easy conclusion. 

In order to verify the proposition (ii) we set 

M = f (x,b) 6R n+m | Ax - b = 0, x^oj 
Df 

and 



G = j(x,b) £ M 
Df 

Obviously, the 3et G 



x>j,...,Xg integer] . 

/V 

is uniformly distributed in M. We assume 



a > sup inf ||x - y || . 
xeM yeG 

If xeM(b), then (x,b)eM and there is a point (x°,b°)6G with 
|| (x,b) - (x°, b°) || ^ a . 

Since x°£G(b°) there is by (i) a point x*6G(b) such that (9) 
holds, for these points. Now one can estimate 

|| x -x*|| 4 || x - x°|| + || x° -x*|| ^ a + 2g ||b° - b II + 2TT 

< a + 2 p a + 2'fc' = 6. 

Df 



(iii) Let {v^,...^ 1 *] be a system of (mixed-) integer generators 

of V and, let V > max ?6, llv 1 !!? be where 6 is a constant 

i=1 

with the property (ii). Using C(b) from (1) we define 



K*(b) = fx 6 R n |x 6 G(b) , x=y+v, y£C(b), ||v||tf^]. (10) 

Df C 

Obviously, K*(b) is compact and it holds that G(b) = K*(b) + V , 
b£B. To show the quasi-Lipschitzian behaviour of K*: B let 

b,b*6B and x eK*(b) be arbitrarily fixed. Then there are points 
y6C(b) and v with ||v||^'£' such that x = y + v. Prom (3) we can 
conclude that there is a point y* 6 C(b*) such that 

II y - y*|| ^ glib - b* || . 

By C(b*) cz M(b*) and proposition (ii) there is a point x*€G(b*) 
with ||x* - y*||^6. Bow one can estimate 
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|| X - x*|| = ||y + V - y* + y* - x*|| ^lly - y*|| + l|v|| + ||y* - x*|| 

^ glib - b* || + X + 6 , 

and the quasi-Lipschitzian behaviour of K* is shown. ^ 

The essential result for a quantitative stability analysis of P(b) 
consists in the following theorem. 



Theorem 2. 

Let (GA) be satisfied and let 9(0) = 0. Then 

(i) the optimal set-function 'Vf/ is quasi-Lipschitzian on B, 
moreover there is a 

A 

function K: B — >2 

yW = K(b) + V s , b£B 

where 

V a = fx £R n | Ax = 0, c T x^0, x^O, Xy,,...,x a integer}; 
3 Df 'is 

(ii) the value function is quasi-Lipschitzian on B. 



guasi-Lipschitzian, compact-valued multi- 



such that 



Proof: We show the first part of (i) and using this we verify (ii). 

Exploiting both the second part of (i) can be shown. 

By assumption, Y(b) ^ 0 for all b 6B. For b,b*6B, xe Y(b) and 
x*6Y(b*), (all this points arbitrarily chosen) and, using G(q) 
and ^ 2 q as defined in (7) with a K: dom Gr — *2 R such that (8) 

holds, we have (x,x*) £G(b-b*). Further, there are points 
(y,u) £ K(b~b*) and (y,u)6V 2s s 110 * 1 that 

(x,x*) = (y,u) + (y,u) , 

II (y,u)|| ^ g || b - b* || + X . 

One easily shows that f e Y(Ay) and u ^^(Au). Since Ay = Au 

T— T- - - 

one has c y = c u and, for the point x = u + y 

Df 

T— » T T— T T- T 

cx = cu + cy = cu + cu = c x* 

holds from which x 6 \p(b*) follows. Now the first part of (i) is 
an immediate consequence of the estimation 

|| x - x II = || y + y- u- yll = ||y-u||^ 2||(y,u)|l ^ 2 g |lb - b*|l + 2X . 
If b, b*, x and x* have the same meaning as before then 

| 9(b) - 9(b*) | = |c T (x -x*)| ^ I(c|l2p ||b -b*il + 2 X (>M ) 
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is valid which implies (ii). 

It remains to show the second part of 



and 



N(b,a) = (x6 G(b) | c T x^a? 
Df 

V = fxfeV I c T x^0 ] . 

Df 



(i) 



Let 



By (GA) the polyhedral cone V has a system of integer generators* 
Since <p(0) = 0, it follows that c^v^O on V, saying that a sub- 
system of an integer generating system for V generates V, too. 
Therefore, V has an integer generating system (independent of 
the fixed c e R n ). On this basis following a similar way as used in 
the proof of Theorem 1. one can construct a quasi-Lipschitzian, 
compact-valued multifunction K: dom IT — > 2 Rn such that 



N(b,a) = K(b,a) + V s V(b,a)6domN 
where V s = [xeV |x /| ,...,x s integer]. 

Let a and p be constants such that 



for all 



one has 



d(K(b,a),K(b* a*)) ^ ct||(b,a) - (b* a*)|| + (3 

(b,a), (b*, a*) £ dom IT. Then for 

K(b) = K(b,cp (b) ) 

Df 



'y(b) = K(b) + V s \/b 6 B. 

Applying (ii) and taking (11) into account we get 
d(K(b),K(b*)) £ a||(b,cp(b)) - (b*,(p(b*))|| + p 



^a(||b - b*|l + | <p ( b) - <p(b*) | ) + |3 
£ a(||b - b*|| + 2 ||c || g||b - b* II + 2x) + p 
= a(1 + 2 ||c || g ) || b - b*|| + 2?a + p 
which completes the proof, u 



We remark that there is only a finite number of different V possi- 
ble to describe N(b,a) if c is allowed to vary* 



The next theorem is devoted to relations between the (mixed-) integer 
problems P(b) and their linear programming relaxations P (b) 

(i. e. the case s = 0; by (p Q (b) and Y 0 (k) we denote the value 
function and the optimal set-function of P Q (b), respectively). 
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As usual in convex analysis we use the notation of the epigraph of a 
r eal -valued function f on R n -, i. e. 

epi f = f(x,x n+ 1 | )eE n+1 xfedom f j . 

Theorem 3» 

Let (GA) be satisfied and let cp(0) = 0. Then 

(i) there is a constant A such that 

d( A V b 6 B; 

(ii) sup £cp(b) - cp Q ( b ) | b 6 B j ^ oo ; 

(iii) the epigraph of the value function <p Q coincides with the 
convex hull of the epigraph of the value function 9 . 

Proof, Exploiting Theorem 2. one obtains the propositions (i) and 
(ii). Applying linear parametric programming one shows (iii) (see 
e. g. Nozi&ka et al. (4974) ) • & 

In the proof of the next theorem we use the following decomposition 

T s 

of x fcR n into subvectors z = (x,j, ...,x g ) fcR and 
y = (x xl T ; the matrix A and the vector c are decomposed 

S-f'1 n t \T 

into A = (Ay(,A 2 ) and p = (c^ o fl ) , q = (c g+/) , • • • » c n ) » 

re spectively . 

Theorem 4. 

Let (GA) be satisfied and let tp (0) = 0. Further, let b*eB be any 
fixed parameter. Then there is a number = l?(b*)>0 such that 

9(b) > cp(b*) - 'O' (I b - b*|| Vb 6B. 

Proof. Defining sets K(b) and V 0 according to (4) we have that 

— — T^n ° 

(5) holds and, that K: B — ► 2 11 is compact -valued and upper semi- 

continiuous. Prom the proof of the second part of the proposition 
(i) in Theorem 2. we know that ^(b)nK(b) ^ 0, for all b 6 B. Now 
let {z z r j be the finite set of all different integer vectors 
z i 6 R s for which there are vectors y 6 R n “ s such that 
x = (z*,y) e K(b*) . Let £>0 be sufficiently small such that the 
implication 

X = (z,y)eu e K(b*) =>ze{z 1 

holds. Then the uppersemicontinuity of K implies the existence of 
a number 6 = 6 (b *)>0 with 

K(b) CDjKtb*) VbeU 6 (b*) 
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and, one concludes that 

b6UgCb*), (z,y)£K(b) z e {z A , . . . , z r j. 

Using the definitions 

<p( A) = inf {q T y I A ? y = A , y^o} , A e dom 9 
Df 

1(b) = {ie | 3(z\y)feK(b)?, 

Df J 

we can write, for all bfeU^Cb*), 

9(b) = inf {p T z x + <p(b - A^z 1 ) ? 
iel(b) 1 J 

where I(b*) = {l,...,rj. 

By Theorem 2. (ii) (the case s = 0) there is a constant number 
Y,| >0 such that 

I <p( A) - <p(b* - A /| z i ) | Y/| II A - b* + A^z i ft , 

for all X for which there is a point y^O with k^y = X and 
all ie {l, ...,rj. Prom the obvious inequality 

9( -A) ^ 9(b* - A^z 1 ) - |9(A) - q>(b* - A 1 z 1 ) | 
one obtains the estimation 

9(A) ^<p(b*- A^ z 1 ) - Y/| li A- b* + A^z 1 1| , (12) 

for all X e dom 9 and i 6 £ 1 , . . . , rj . 

If now be.U^b*), then there is an index i Q eI(b) such that 
9(b) = P T z 0 + <p(b - A^z 10 ) which yields by (12) the estimate 

9(t)^p T z l0 + <p(b* - A^z 10 ) - Yjlb - A^z 10 - b* + A^z 10 )! 

2<p(b*) - Y/j II b - b* || , 
for all b&U 6 (b*). 

On the other hand, if b^U^b*) then, using constants y» ^ such 
that 

|9(b) - 9(b*) | ^ Y lib - b* II + ^ V b,b*6 B 
(which exist by Theorem 2. (ii)), one obtains 

9(b) £ 9(b*) - | 9(b) - 9(b*)| ^(b*) - (y +^)||b - b*|| . 

Setting = if(b*) = max iYvp Y + ^6”^ j the proof is completed. ^ 




13 



3. Applications to Exact Penalty Functions for Pure- and Mixed- 
Integer Linear Optimization 

The behaviour of penalty methods essentially depends on the involved 
penalty functions. In integer programming a particular difficulty 
appears in finding an appropriate penalty term. Using the results of 
the previous section we point out that there exist exact penalty 
functions for pure- and mixed- integer linear optimization problems 
with a structure allowing the design of solution methods. 

Let P(b) be defined as before. For a fixed right-hand side b*eB 
of the linear constraints and a number r^O we associate with 
P(b*) the mixed- integer problem 

P* inf {c T x + r||b* - Ax || | x £0, ...,x s integer]. 

By cp* and 'Vp* we denote the corresponding optimal value and the 
optimal set, respectively. 

Theorem 5* 

Let (GA) be satisfied and, let cp(0) = 0. If b*6 B is fixed and, 
if r is a sufficiently large number, then 

Y(b*) = 'vp* and cp(b*) = cp* 

hold. 

Proof. From Theorem 4* we can conclude that 

9(b)»«p(b*) - tfflb - b*|| VbeR 111 

where <p(b) = + eo if b $B. For r >'& and x>0, x^, . ..,x g 
integer such that Ax 4 b* we obtain the estimate 

c T x + r ||b* - Ax|| ^ cp(Ax) + r||b* - Ax II 

>cp(Ax) + i7||b*- Axil ^<p(b*) 

from which the proposition is an immediate consequence. ^ 

We consider now the pure-integer problem, (i. e. s = n) 

P(b) inf {c T x | Ax = b, x^O, x integer] 
for a fixed b €B. Let 

L(*,u,r) - (<= -A) 1 * - b » 2 - 

Df 

Theorem 6. 

Let (GA) be satisfied, cp(0) = 0 and b fcB a fixed parameter. Then, 
for any u £R m there exists a number r(u)^0 (not depending on b) 
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such that 

L(x*,u,r) = min £L(x,u,r) | x>0, x integer j 
holds if and only if x*£'y(b), for all r^r(u). Moreover, 
L(x*,u,r) = cp(b) . 

Proof. On account of B^(xeR n | x integer j, Theorem 2. yields the 
existence of a constant number g>0 such that 

| <p(h) - cp(b» )| ^5llb-b'|| Vb,b'6B. 

Further, lib - b f || £ ||b - b* || 2 , for all b,b* 6 B. For an arbitrarily 
fixed u€R m we put 

r(u) = 1 + g + ||u|| . 

Df 

Now, let x>0, x integer and Ax ^ b, and let r^r(u). Then 
L(x,u,r) = c T x + u T (b-Ax) + r||Ax-b|| 2 ^ 

>, cp(Ax) + u T (b-Ax) + r||Ax-b||^ 

^(p(b) - | cp(b) - 9 (Ax) | + u T (b - Ax) + r||Ax - b|l ^ 

^9 (b) - gj|b -Ax|| - ||u|| ||b -Ax|| + r||Ax - bl|> 9(b), 
which the propositions of the theorem are simple consequences of.^ 

Remarks . 

Theorem 6. shows that L(x,u,r) yields a penalty function (for the 
pure integer problem P(b)) which, obviously, possesses ^nic.er' 1 
properties than the one considered in Theorem 5. 

Bounds for the "sufficiently large" numbers r occuring in both the 
Theorems 5. and 6. can be derived using estimates of the constants 
taken from the Theorems 2. and 4. or of these given by COOK et al. 

(1 986) . 
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1. Introduction 

Given the unconstrained minimization problem 

f(x) » min! Sot. xCI? (1) 

and the nonlinear programming problem 

f(x) a min! Sot. x€G 

r 7 (2) 

where G = | x,: (x) ^ 0 j=1,o..,mj. 

To obtain a globally and locally superlinearly convergent serial 
method it is possible to combine a globally convergent method I 
and a locally superlinearly convergent method II for a serial 
computer* 

These hybrid methods can be regarded as realizations of the following 
general principle: . 

After having generated the iterates x k resp. z k = (x k ,u k ) (an 
approximation of the KUHN-TUCKER-point /=* (x*,u* ) by one step of 
the method I and II, respectively, we decide by use of the conver- 
gence Tests I and II, respectively, whether the generation of 
x k+1 resp. z k+1 by method II is advantageous or not. 








Examples of this principle are: 
for problem PI : 
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method I: Method of deepest descent 

method II: Newtons method 

for problem P2 : 

method I: Penalty-Barrier-method 

method II: Wilsons method 

The aim of the paper presented is to transform the prescribed 
principle on parallel computers. The method obtained has the 
important advantage that the convergence i3 globally and locally 
quadratic ally, independet from the dimension of the considered 
problem. This is a different to the sequential secant method with 
the trend to linear convergence, if the dimension is increasing. 

The extension of the prescribed technique on the treatment of 
problem (2) is obvious. 

2. Notations and preliminaries 

Denote a set of n+2 points by x^ * fx^* \ • . . ,x k,n+2 J. 

•.lri.1 

In the following algorithm when we write, that x is computed 
parallely with the use of some methods, it means that each x * 

(i=1 , • • . ,n-f-2) is computed by the i-th processor with the use of 

lr -i 

this method on the basis of the prints x , • . .i-1 ,i+1 , • • .n+2. 

We indroduce METHOD 1, METHOD 2, TEST 1 and TEST 2 in the following 
way : 

METHOD 1 ; 

Given n+1 points x^ , • • • ,x n+1 • Choose £ with 0 ^ 6 < 1 and § vilth€«€ 

Set 1=0 

SI : Set y 1 = x 1 

y* = x 1 + (j) 1 (x^x 1 ) (i=2, o . . ) 

S2 : Compute the direction g as a solution of the linear system 
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where f^fCy 1 ) (i=1 , . o . ,n+1 ) 

1 1 *1 
S3: Determine the greatest value of oc « (j) J (o=1 ,2, •<>•), 

such that 

fCy 1 - 0£g)^f( y 1 ) - 0 C 6 IlgH 2 

S4: If an oC exists, such that oC>£ , then set 

x 11 * 2 = - ol g 

else set l=l-hl and goto S2. 
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Test 2s If |Vf(x k+1 » i H * c. min f(x k,i ) 
for all i, then set 

6 s ^ • C^) , 

x k4-1,i = w k+-1,i 

and METHOD = METHOD 1. 



3® Algorithm 

SO: Choose a set of n-t-2 initial points 

x 0 ’ 1 ,...,* 0 ’ 11 * 2 5 £ > 0 ; C r c 2 e (0,1 ))§>0 

k-th step of the iteration (k>Q); 

SI : Determine x^ 4 ^ parallely with the use of METHOD® 

S2 : TEST 

S3: Set k»k+1. Goto SI. 

4. Properties of METHOD 1 and METHOD 2 

For the proof of the global convergence of METHOD 1 we consider the 
generated sequence fw^j with the initial points 

w" n , w" n+1 w° 6 B? 1 . 

At first the following conditions are formulated® 

Condition (Cl): We say that the function f(x) fulfilles the 

condition (Cl), if it holds: 

(i) s 3T i f(x)£T V xerf 1 

(ii) : for a given x^R?: W(x°) ={x: f (x) s£ f (x°) j is closed 

(iii) s f (x) is differentiable on W(x°) and for all x,y£W(x°) 

with the property 

[x,y]„ fz : z -ax+(1-A)y* AeO,ljc W(x°) 

the relation 

IIVf(x) - Vf(y)| £ Llfx-y/J (1>0) 

is satisfied. 
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Condition (02) t For a given we say that n Vektors 

d* » - w*" 1 (i»k-1,. 0 .,k-n) are w-non3ingular, if 



det 



— 

VT 




) ^ 60 



Theorem 1 : We assume, that for a given w° the function f(x) 

fulfiles the condition (01) and that n+l points (i=0 n) 

fulfil condition (02) for all k (independently from k). 

Then the sequence is well-defined for all k and it holds 



lim inf || g^|| = 0 and lim inf ljVf(w)ll = 0. 
k-» k-»°° 

Before we discus properties of METHOD 2 some conditions for the 
minimum point x* and the function f(x) are formulated. 



Condition (C3)s We say that the condition C3 is satisfied, if for 
the minimum point x* € R 11 and for some ball S* = S( x* 9 <f) it 
holds : 

i) f is twice differentiable on S 

ii) Vf(x* ) » 0 

iii) the Hessian H(x* ) is positiv definit 

iv) H is Lipschitz-continuous on S* with a constant L, i.e. 

II H(x) - H(y)M L || x - y/| Vx.ye S* 

If C3 is satisfied then the point x* is a so-called regular 
minimum-point. 

Theorem 2 s Let x* be a regular minimum-point and let x~ n , .. o ,x 0 ) 
be a set of points containing in a suitable neighboorhood SCx*,*/ 1 ) 
of x * and satisfying Cl. 

Then the sequence {x k j generated by METHOD 2 is well defined. 

Furthermore, either the algorithm terminates after a finite number 

of iterations at the regular minimum point x N = x* of f or x^ 

is infinite and lim x^ = x*. 

k-*®o 

The rate of convergence a 6 is the positive root of. 36 * 




21 



Sketch of the proof : 

It is possible to prove, that under the required assumptions the 
generated matrix G is a strong consistent approximation of 
H(x k ). Therefrom and from the assumption, that the iterates 
x k ,x ic " 1 ,...,x k " 1 are contained in the ball S it is 

possible to derive the nonsingularity of H(x k ) which implies the 

k-t-1 

existence of x and the relation 

II x M - x*IU K^IIX*-* - x* II) 9 X k - x'll . (8) 

i=0 

If the inequality 

k(£:iix*-' - x*n) 

i=0 

holds, then the point x k ‘ f ** belongs to the ball S (x* 9 cf)o It is 

k # 

clear, that lim x = x . 

5> Convergence Analysis 

""’k 

In order to investigate convergence properties of the iterates x 
generated by the hybrid algorithm we define the index set 

K- =fk: x k+1 is generated by METHOD ij 

* — k+1 

For all remaining iteration indices the iterates x are 

determined by METHOD 2. 

Now we consider the index set corresponding to METHOD 1* 

With the presented theorems we can conclude that is finite 
because both TEST 1 and TEST 2 lead to the locally convergent 
method for every safficienthly large k. 

Y/e summarize the behaviour of |x k j in the following theorem* 
Theorem 3 

We assume, that with the points x 0,i (i=1 , . . • ,n+2 ) and the level 
sets W(x 0,i ^ the condition (Cl) is satisfied. 

Furthermore the condition (C2) may be satisfied for every subset of 
n-fl points x k,i of x k independently from ko 
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If additionaly eaoh minimum point of f is regular then the generated 
sequence [ x^ J has the f ollowing properties : 

i) lim x k,i = x* 

ii) the rate of convergence is at least R - quadratic ally* 

Proof : 

The first part of the theorem is obvious from the remarks above* 

The second part required considerations on the R-order of sequences 
[x * j coupled by using of a parallel algorithm* 

If we transform the estimation ( ) on this situation, the system 

of inequalities 

|x k+1 » 1 -**||ik 1 21 flx k ’ i -x*ll IU k ’;U*|| (i=1 , . o . ,n+2) 

3*i 

anses in a natural way© 

It is possible to write the relations (8) in the form 

]| x kf 1 'i-x* II ^ ^ 21 fl ll* k,1 -x* T 11 . (9) 

j*i ,1=1 

where the coefficients m il of the matrix are 0 or 1 • 

For first order difference inequalities of this kind SCHMIDT (/2/) 
has proved, that the R-order of the underlying sequences of iterates 
is at least equal to the spectral radias of the matrix of exponents 
if a positive eigenvector belonging to the spectral radius exists* 

For the application of SCHMIDT’S result we consider the structur 

of the matrix M^. In each row of these matrix there are two elements 

equal 1, the other elements are equal 0* It is immediatly obvious, 

T 

that e = (1,..o,1) is an eigenvector of M* and 2 is the eigenvalue 
belonging to it© 



Remark; 

The quadratically rate of convergence diffus from the result in the 
serial case, where dB is the positive root of the equation 



ae 



n*1 



n 



de + 



1 . 
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Depending on the dimension n this root has the following values 

n ag 

1 1.618030 

2 1.465576 

4 1.324718 

5 1.285191 

and it holds 

lira ag(n) « 1 
n-*«o 

6. Concleeding remarks 

It is possible to extence the idea prescribed above to realize a 
whole class of hybrid methods on parallel computers. Especially 
using of parallel stochastic search methods as METHOD 1 and 
derivative-free-methods as METHOD 2 is advantageously. 
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Continuous Generalized Gradients for Nonsmooth Functions 
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At present there exist many generalizations of the concept of 
gradient for nonsmooth functions. We mention only those of R. T. 
Rockafellar [1], F. Clarke [2], J. Warga [3], J.-P. Penot [4], 

B. Mordukhovich and A. Kruger [5], V. F. Demyanov and A. M. Rubinov 
[6]. But all these generalizations inherit the main cause of non- 
smoothness - the absence of differentiability which gives rise to 
the discontinuity of corresponding mappings. 

In what follows we try to restore the lost continuity (at least 
for some classes of nonsmooth functions) . 

Let f be a real-valued function defined on IR n . We say that f is 
codif ferentiable at xG3R n if there exist two convex closed sets 
df(x) clR n+ ^ and df(x) cIR n+1 such that 

f(x+A)=f(x)+ maxf a+ (v, A) ]+ min[b+(w,A) ]+o(A,x) 

[ a, v] Gdf (x) [ b , w] Gdf (x) 

where — -r~>0, for all AG3R n . 

a a+o 

The pair of sets Df (x) =[ df (x) ,df (x) ] is called a codifferential 
of f at x; df(x) is called a hypodif ferential and df(x) a hyperdif- 
ferential of f at x. It is not difficult to see that the class of 
codif ferentiable functions coincides with the class of quasidiffer- 
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entiable functions [6]. It is also clear that a codifferential is 
not uniquely defined. 

If there exists a codifferential of f at x of the form Df(x)= 

=[df (x) , (0 n+ ^ } ] then we say that the function f is hypodif ferentiable 
at x; and if there exists a codifferential of the form Df (x) =[ (0 n+1 } , 
df (x) ] then f is called hyperdif ferentiable at x. 

If f is codif ferentiable in a neighbourhood S of x then a codif- 

JR n+ ^ JR n+ ^ A 

ferential mapping Df:S -2: x T (where 2 is the power set of A) 

is defined. We say that f is continuously codif ferentiable at x if 
f is codif ferentiable in a neighbourhood S of x and there exists a 
codifferential mapping Df which is continuous (in the sense of 
Haussdorff) at the point x. 

- 1 

Note that all C -funtions, all convex and concave functions, all 
functions of maximum and minimum (over a compact set of C functions) 
are continuously codif ferentiable . 

Calculus of codifferentials can be constructed: the set of con- 

i 

tinuously codif ferentiable functions (CC -functions) is a linear 
space closed with respect to all "smooth" operations and (more im- 
portantly) those of taking minimum or maximum over a finite number 
of functions. 

Thus 9 the continuity of generalized gradients is restored by 
introducing a new concept (that of codifferentiability) . We can 
apply this new concept to construct a new family of numerical meth- 
ods to optimize CC 1 -functions incorporating (with natural adjust- 
ments) all existing algorithms for quasidif ferentiable optimization 
(see [ 7 ] and [ 8 ] ) . 

It is worth mentioning that as it turns out the effectiveness 
of the "linearization method" of B. N. Pshenichny [9] for minimax 
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problems is due to the fact that max-functions are continuously 
hypodif ferentiable . 

A detailed report will be published elsewhere (see, e. g., [10]). 

Remark : The concept of codifferential allows to construct first- 
order approximations to sets described by codif ferentiable func- 
tions. This seems to be extremely promising from computational stand* 
point . 
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Summary . A brief report is presented on recent developments in outer 
cut methods for solving certain classes of constrained global optimiza- 
tion problems. First, two theorems are stated each of which is help- 
ful to assert convergence of a basic concept. Nonlinear cuts and un- 
bounded feasible sets are admitted. A broad class of new cutting plane 
methods can be deduced. Moreover, a practically useful constraint drop- 
ping strategy is given and, finally, some questions of numerical imple- 
mentations are discussed including the computation of new vertices gen- 
erated by a cut and the identification of redundant constraints. 



1. Introduction. Cutting plane methods have been used in different 
fields of optimization for some time. A cutting plane is a linear con- 
straint that is used to reduce a certain set in such a way that does 
not exclude optimal feasible points of an optimization problem. In the 
sequel, we will only consider cuts that never cut off parts of the 
feasible set: outer cut methods. However, nonlinear cuts will be 
admitted. 

We have in mind continuous global optimization problems where nonglobal 
local minima occur. Let throughout this paper f : R n — ► R be a conti- 
nuous function and let D denote a closed subset of R n . Furthermore, it 
is supposed that min f(D) : = min {f(x) : x e D} exists. We state our 

problem in the form 



global min f (x) . 
xeD 

Integer programming problems , where cut methods also belong to the basic 
tools, will not be discussed explicitely here. Note however, that integer 
programming is closely related to certain global optimization problems 
(cf . e.g. [10] , [24] ) . 

Other fields, where cutting plane methods have been extensively inves- 
tigated, are convex programming and local, possibly nonsmooth optimi- 
zation. 

The first methods applied in convex programming (as well as in certain 
fields of approximation) were the so-called KCG algorithm (cf . [1] , [17] ) 
and the supporting hyperplane approach (cf. [32]). Since however, the 
numerical behaviour of these methods was not very satisfactory (low 
convergence rate, stability problems near a solution) , several modifi- 
cations and combinations with other devices were introduced, e.g. line 
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searches, quadratic approximation etc.* Today, modified cutting plane 
methods belong to the major tools in nonsmooth optimization (cf. e.g. 

[8] , [9] , [18] ) and references given there) . 

Another challenging question with outer cut methods has always been the 
question of constraint dropping strategies. Since, in the basic version 
of such a procedure, in each iteration a new constraint is added to the 
existing set of constraints but no constraint is ever deleted, the size 
of the problems to be solved increases from iteration to iteration. 
Therefore , much effort has gone in the development of devices that allow 
to drop certain previous constraints (e.g. [6] , [8] , [9] , [10] , [12] , [14] , 

[15], [16], [28]). Of course, the most satisfactory results were obtained 

for convex and local optimization where the number of constraints to be 
considered in each iteration is bounded (e.g. [9], [18] and references 

given there) . 

However, these nice results concerning speed, stability and constraint 
dropping obtained in local (or convex) programming could not been 
carried over to global problems, since, as a rule, local analysis fails 
in global optimization. 

Note as well, that outer cut methods including certain constraint drop- 
ping strategies are strongly related to duality and decomposition (e.g. 
[2] , [20] and references given there) . 

In global optimization, outer cut methods belong to the major tools for 
solving some basically important classes of problems, the most prominent 
example being the class of concave minimization problems, where f is a 
concave function and D is a convex set (cf. e.g. [11] , [26] ) ; for recent 
surveys, see [13], [21], [22]). General theories of outer cut methods 

that considerably simplify and extend previous theories have been deve- 
loped recently in [14] , [29] and modified and generalized further in 

[15] . Moreover, new results on certain numerically important questions 
arising from these methods were recently obtained in [16] . 

It is the purpose of this paper to briefly summarize these results. 
Furthermore , some modifications will be given and several open questions 
for further research will be stated. 

In Section 2, the basic concept and two associated theorems will be pre- 
sented. Section 3 contains a practicable constraint dropping strategy. 
In Section 4, a general class of cutting plane methods is illustrated 
and the final Section 5 is devoted to questions of how to solve the 
subproblems arising in each iteration and how to identify redundant 
constraints . 




30 



2. A Basic Concept 



Consider the problem 

global min f (x) , (1) 

X£ D 

where f : R n — * R is continuous and D is a closed subset of R n . Suppose 
that min f(D) exists. Then, an outer cut concept proceeds as follows. 

Choose a closed set D 1 c R n satisfying D^d D. Set k ♦— 1 . 

Iteration k (k = 1,2,...): 

Solve the "relaxed problem" 

ir 

(P. ) global min f (x) obtaining a solution x . (2) 

** D k 

k k 

If x £ D, then stop: x solves problem (1) . Otherwise construct a con- 
straint function 1^ : R n — * R satisfying 

l k (x) £ 0 Vx £ D, (3) 

l k (x k ) > 0. (4) 

Set D k+1 = fl {x £ R n : l k (x) £ 0} , k «— k+1 and go to the next iter- 



It is supposed of course, that the subproblems (P^) can be reasonably 
efficiently solved (cf. Section 5). The set 

H k : = {x £ R n : l k (x) = 0) (5) 

k 

strictly separates x from D and is called a "cut'.' In most applications 
is a polyhedral set and we have 

l k (x) = (a k ,x) + b k , (6) 

where a k £ R n , b k £ R and (• , •) denotes the inner product in R n . In 

this case, H k is a hyperplane and the method is said to be a cutting 
plane algorithm. 

Two recent results on the convergence of such an outer-cut procedure 
will be presented below. The first result deals with cutting plane 
methods and compact sets D k . 
k 

Let H k = {x : (a ,x) + b k l , k £ N, define a sequence of hyperplanes. 

Then, {H.} is said to converge to a hyperplane 



H = fx : (a,x) +b},a£R,b£R (notation: H, — ► H) if we have 

k k 

a k — * a, b k — ► b. 
k-*~ k-*» 
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Theorem 1. (Tuy [29] ) . Within a cutting plane procedure let D x be com- 
er ]r 

pact and suppose that for each convergent subsequence fx M l of fx [ 

satisfying x q — ► x t D, H — ► H, we have that H strictly separates x 

a a . 

j# 

from D. Then, every accumulation point of (x } solves (1) . 

The following result generalizes Theorem 1 and a theorem presented by 
Horst, Thoai and Tuy [14] , where a uniform Lipschitzian property of the 
cut functions 1^ was required. Moreover, unbounded feasible sets D and 
nonlinear cuts are admitted. 

Theorem 2. (Horst, Thoai, Tuy [15]). Within the outer cut concept pre- 
sented above assume that 

(i) 1^ is continuous for each k = 1,2,... and for each convergent sub- 
sequence fx q }c{x k } we have 1 — >1 where 1 : R n — * R is continuous; 

3 g 

(ii) for each convergent subsequence fx q l c {x^l satisfying 

x q — ► x, 1 (x q ) — ► l(x) = 0 we have x e D. 

3 3 s 

Then every accumulation point of the sequence (x ) solves (1) . 

In contrast to previous rather involved theories (e.g.[6], [12], [14]), 

Theorem 2 seems to reduce the matter to a basic and simple result. It 
has been shown in [15] that a broad class of new cutting plane algo- 
rithms can be derived that includes known procedures as special cases. 
Moreover, several constraint dropping strategies can be deduced. The 
proof of Theorem 2 is very easy: 

Let x^ — * x. Then, by (3) and (4) we see that 1 (x* +1 ) £ 0,1 (x 1 ) > 0 Vq, 
3 3 3 

hence, for q + «*, we must have l(x) = 0. From (ii) it follows then that 
— k 

x £ D. Finally, it is easily seen that, by f (x ) <; min f(D) and by con- 
tinuity of f, x e D implies f (x) = min f(D). 



a. Constraint Dropping 

Let 

H k : = (x e R n : l k <x) S 0) . 

Then, in the basic concept described in Section 2, we have 
D k+i - D i n ( ^ H i>- 

i = 1 



(7) 



try to drop certain constraints defining , i.e. we try to find a 



We 

set of indices 1^ c (l,2,...,k) such that 

Vi - D i " ( n V 

l8I k 

defines a convergent outer cut procedure. 



(8) 
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In their outstanding paper [6] , Eaves and Zangwill presented a general 
theory of cut procedures and constraint dropping strategies. Cuts are 
defined via a set valued mapping called a cut map that has to satisfy 
certain assumptions concerning regularity and separation. Separation is 
defined by a real-valued function called a separator that, in a very 
general sense # describes a distance from its argument to the set we want 
to attain to by the cut procedure. 

More recently, it was shown by Horst, Thoai and Tuy [14] that, in the 
practically most interesting case of cutting planes, the cut map/separator 
theory can be considerably simplified: new proofs of basic results are 
given and several classes of constraint dropping strategies are proposed 
that are of interest in global optimization. 

In a later paper [15] , it could be shown by the same authors that all 
these constraint dropping strategies remain even valid if nonlinear cuts 
are applied. Moreover, it it demonstrated, that the earlier and some new 
results on outer cut methods can be obtained by a very straightforward 
access via Theorem 2 that is considerably less involved than the cut 
map / separator theories. 

Below, we present as an example, that subclass of constraint dropping 
strategies included in the development given in [15] that after first 
preliminary numerical experiences seems to be promising for practical 
use . 

Let p : — > R be a continuous function satisfying 

p ( x ) >0 Vx £ D x \ D , p ( x ) = 0 Vx £ D. (9) 

p(x) is frequently called a penalty function . 

Many penalty functions are known in nonlinear programming. Let, e.g., 

D = fx £ R n : g i (x) £ 0 (i = l,...,m)}, 

where g^ : R n — ► R (i = 1,...,) are continuous on D^. Then, it is well 
known that all functions of the form 



m 

p (x) = S [max 



i = 1 

are penalty functions. 

Furthermore, let fa^j 

quence satisfying ^ 

£ . . : = a . - a . , 
13 i 3 

Take, for example, 



(0, g i (x) ) ] p £ 1 (10) 

c R + be an arbitrary monotonically decreasing se- 

— ♦ 0 (notation: a \ 0) and let 
-*x> 1 

i * 3- 

= (a) 1 where a £ ]0,1[. 



( 11 ) 
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Theorem 3, Let p(x) be a penalty function as defined in (9) and let 
be a double-indexed sequence as defined in (11). 

Assume that in the outer cut concept the sequence of cut functions sa- 
tisfies the separation requirements (3) and (4) and the assumptions (i) , 
(ii) of Theorem 2. Then 

D k+1 = D 1 D ( n hT) k = 1 , 2 ,... ( 12 ) 

ieI k 

with 

I k = {i : X S i <; k, pfx 1 ) * e ik ) (13) 

defines a convergent outer cut procedure, i.e., every accumulation point 
of fx } solves problem (1) . 



Remarks . Note that, by e kk = 0, we always have k e I k - Moreover, since 
for fixed i, e^ k is monotonically increasing in k, we have that i t I k 
implies i t I g Vs > k: a dropped cut remains dropped forever. 

For the same reason, a cut retained at some iteration may be dropped 
at a subsequent iteration. 

Note as well that typically the penalty function's values p(x x ) de- 
crease when the iteration sequence approaches the feasible set D. 

The results of numerical tests we are engaged in will be reported 
elsewhere . 



Open Questions. 1. It is not yet known if there are interesting classes 
of multiextremal global optimization problems, where it can be shown 
that the number of constraints to be retained at each iteration is 
bounded . 

2. An outer cut method with constraint dropping strategies that allows 
to use nonlinear cuts, conceptually could be applied for solving non- 
linear programming problems having a large number of constraints: Solve 
the problem while considering only a small part of its constraints ob- 
taining a solution x 1 . Consider the constraints not yet taken into 
account that are violated by x as potential cut functions and apply an 
outer cut procedure with constraint dropping strategy. Each subproblem 
actually consists in solving the original problem under a subset of its 
constraints .Can such a procedure be organized in a way to become compe- 
titive with other methods used in large-scale optimization? 
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4. Cutting Plane Methods. 

Let in the outer cut concept be a polyhedral set and consider affine 
cuts 1^. Let 

D = lx t R n : g (x) S 0) (14) 

where g : R n — » R is convex (often: g(x) = max g. (x) , where g. : R n — * R 

i e I 1 1 

is convex, i € 1, I c N, HI finite). 

Cutting plane methods for solving the resulting problems having convex 
objective function were given e.g. in [1] , [6] , [8] , [9] , [12] , [17] , [18] , 
[23]. Concave programming was treated , e . g ., in [11], [13], [27], [29]. 

General global optimization concepts were treated in [14] , [15] . 

Let the cuts 1^ be defined by 

l R (x) = (p k , x - y k ) + 3 k , (15) 

k n k 

where p z R , y e and $ k z R are suitably chosen. Note that well- 

k k 

known classical methods are obtained by specific choices of p , y , (5^. 
Set 

p k z 9g(y k ) , = g(y k ) , (16) 

where 8g(y ) denotes the subdifferential (or e-subdifferential, cf.[23]) 

- . k 

of g at y . 

k k 

Then, for y = x , we obtain the KCG procedure [1] , [17] . The supporting 

k 

hyperplane method results if y is chosen to be the intersection of the 

Ok Ok 

line segment [y ,x ] joining an interior point y and x with the boun- 
dary of D. 

The general approaches of [14] , [15] allow to construct a broad class 

of new cutting plane methods satisfying (15) , (16) that include previous 
algorithms as special classes. Let conv f denote the convex hull of a 
set A c R n . 

Theorem 4 . Horst, Thoai, Tuy [15]) Let in problem (1) the feasible set 



D have the form (14) and assume that D 



{x_e R 



(x) < 01 is not 



convex, compact subset of D and 



choose in (15) 

y k e Y k : = conv (K U (x k ) ) \ D°, p k £ 8g(y^ \ (0), P k = g(y^, (17) 

— 

where x z argmin f(D k ). 

Then the separation requirements of the basic outer cut concept and the 



assumptions of Theorem 2 are satisfied. 



v 

The set Y is illustrated in Fi 
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Remark . Theorem 4 implies in particular that, whenever there is an al- 
gorithm for the relaxed problems (cf. Section 5), then the constraint 
dropping strategies resulting from Theorem 3 can be applied indepen- 
dently of the form of the objective function. 

Open Question. It is not yet clear which one of the many implementable 
algorithms that fit with the result of Theorem 4 should be chosen for 

solving certain classes of global optimization problems. First prelimi- 

k 0 

nary computation on convex minimization suggests to choose z e D and 

0 k 

to apply the supporting hyperplane method with y replaced by z in 

k k k 

such a way that the intersection point y of [z , x ] with the boundary 

k k k 1 k k 

of D approximately bisects [z , y ], i.e. y « ^ + x ). 




Fig. 1. The set Y of Theorem 4 

5. On Solving th« Relaxed Problems 
5.1. New Vertices 



In global optimization, outer cut methods have been applied for solving 
several types of problems, most of them belong to the class of concave 
progamming problems, where in (1) f is concave and D is convex and 
compact (cf .e.g. [7] , [11] , [13] , [20] , [21] , [22] , [25] , [26] , [29] , [30] , [31] ) . 
Except for very few examples (cf . e . g. [27] ) , the cuts always were linear 
i.e. cutting plane methods were applied. 

One starts with a simple convex polytope that, in most cases, is an 
n-simples or an n-rectangle (for a discussion on how to find D^, see e. 
g. [13]). Since f is assumed to be a concave function, we always have 
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min f(D k ) = min f(V(D R )), (18) 

where V(D k ) denotes the vertex set of D^. Thus, the relaxed problems 
(p^Jare solved by comparing the objective function's values at the ver- 
tices of . Then obviously, the computationally most expensive part of 
these algorithms consists in the calculation of the new vertices 
created by a cut: 

Given the vertex set of a convex polytope P and given a hyperplane H 
created by the cutting plane algorithm, find the vertex set 
V(S) of S : = P H H. 

Since obviously, S is a convex polytope, one comes to the basic problem 
of finding all vertices of a convex polytope given by a system of fini- 
tely many inequalities. This problem has been discussed extensively in 
the literature and several algorithms are available (cf . e .g. [3] , [4] , [5] , 
[19] and references given there.) However, numerous examples we calcu- 
lated and the following considerations show that it is advisable to ex- 
ploit specific given properties instead of applying one of the algo- 
rithms for finding all vertices of a convex polytope given by a system 
of linear inequalities. Let 

H = { x £ R n : l(x) = (a, x) +3 = 0}, a £ R n , 3 e R. (19) 

Define 

V + (P) = {v £ V(P) : h ( v) > 01, (20) 

V - (P) = (v e V(P) : h ( v) < 0). (21) 

Note that, in contrast to the setting of the more general problem of 
finding all vertices of a polytope given by linear inequalities, in our 
case V + (P) and V (P) are known. Moreover, for all examples calculated 
with a cutting plane procedure for solving concave minimization prob- 
lems the number 

min ( |V + (P) | , |V~ (P) 1 1 , (22) 

where |V| denotes the number of elements of a finite set V, was always 
considerably smaller than the number |V(S)| of generated vertices (for 
a detailed report on numerical experience, see [16]). 

The algorithm we propose for finding V(S) is based on the following 

Proposition 1. [16] We have that w £ V(S) if and only if w is either a 
vertex of P lying in H or a point where an edge [u, v] of P, u £ V (P) , 
v £ V* ( P ) intersects H. 

Let |V| = min [ |V + (P)|, |V (P)||. For each u £ V denote by E(u) the set 
of half lines emanating from u, each of which contains an edge of P. The 
algorithm essentially consists in computing for each eeE(u), the inter- 
section w of e with the hyperplane H and a feasibility check on w £ P 
or w t P. Consider the constraints defining P and let J(u) be the index 
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set of those constraints that are active (binding) at u. For each sys- 
tem of n linearly independent constraints 



(a , x) + b. <; 0, i. £ J(u) (k = l,...,n) (23) 

x k 

add the new constraint 1 (x) £ 0 or 1 (x) £ 0 (depending on V). Then the 
necessary calculations can easily be carried out by pivoting on a sim- 
plex tableau corresponding to (23) and l(x) £ 0 (l(x) £ 0)) and a sui- 
table bookkeeping. For details, see [16]. 

Note that, for nondegenerate vertices u, there is only one system (23) . 

A detailed theoretical and numerical comparison with previous procedures 
proposed by Thieu, Tam and Ban [26] and by Falk and Hoffman [7] , [11] is 
reported in [16] . It turned out that for very small problems with 
| V ( P ) | £ 100, one should use our method or, in a very straightforward 
way, for each pair u e V (P) , v e V (P) , determine the intersection w 
of the line segment [u, v] with H and decide by the rank of the system 
of inequalities that are active at w, whether [u, v] is an edge of P or 
not. For larger problems our algorithm should be used. 



5.2. Redundant Constraints. 



Redundancy of constraints in linear programming has been discussed by 
many authors (cf. e.g. [5] and references given there). However, by con- 
siderations similar to those of the preceding subsection, in our case, a 
procedure for finding redundant constraints should be based on the 
knowledge of the vertex set V(P). A detailed presentation of suitable 
notions and characterizations of redundant constraints via V(P) is 
given in [16] . 

We briefly mention the most useful result. 

Let 

P = {x £ R n : g k (x) £ 0, k £ 1} (24) 

be a convex polytope defined by the linear qualities g k (x) £ 0, k 8 I, 

and let P be the convex polytope 

P = P n (x e R n : l(x) £ 0), (25) 

where l(x) £ 0 is an additional linear constraint. 

Definition 1. A constraint g^(x) £ 0, k £ I is strictly redundant for 
the polytope P (relative to g..) if there is a j e I \ { k } such that we 
have 

F k ; = P i n (x £ R n : g k (x) = 0 1 c ( x £ R n : g i (x) >0), 
where P^ : = (x £ R n : g^(x) SO, i e I \ (j). 



( 26 ) 




38 



Theorem 5. Horst .Thoai .de Vries [161. a) A constraint g k (x) £ 0, k e I r 
is strictly redundant for P relative to 1 if and only if 
g k (u) < 0 Vu c V(P) \ V + (P). 

b) Every constraint that is strictly redundant for P (relative to any 
g^) is strictly redundant for P relative to 1, 

Remark . By Theorem 5, at an arbitrary chosen iteration of the cut pro- 
cedure, all strictly redundant constraints can be identified. However, 
there can be redundant constraints that are not strictly redundant. A 
definition of redundancy is obtained, if in (26) the strict inequality 
g^ (x) > 0 is replaced by g^ (x) £ 0. A characterization of redundancy 
relative to 1 that uses only known vertex sets is also given in [16] . 
However, if one wants to eliminate all constraints that are redundant 
but not strictly redundant by means of this characterization, one has 
to carry out the corresponding investigation at every iteration of the 
algorithm (cf. [16]). Therefore, in our cutting plane code for solving 
concave minimization, only strictly redundant constraints are elimi- 
nated in some iterations. 
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1 . Introduction. 



Let X and Y be compact spaces and f be from CCX x Y) C the 
space of all continuous real-valued functions on X x Y). Consider the 

"max min" problem C ) of maximizing C over X ) the function 

min < fCx,y) : y e Y >. A solution to this problem is every pair 
Cx 0 ,y 0 ) e X x Y for which 

f<x o’ y o > “ min < fCx o> y> : y « y > = 

max < min < fCx,y) : y e Y > : x e X > 

It is clear that every such problem has at least one solution. 

Ve show here that for a large class of compact spaces X and Y 

C including all metrizable and some nonmetrizable compacta ) there 
exists a dense and G^ subset A of CCX x Y) at each point of which 

the corresponding maxmin problem C P^ ) has unique solution. 



2. Definitions and notations. 



First we remind some definitions. 

2. 1 . Definition. A multivalued mapping F : U 1 > U 2 from the 

topological space U 1 into the topological space U 2 is said to be 
upper semi-continuous at the point u Q e U ± if for every open set V 
in U 2 which contains FCu 0 > there exists a neighbourhood W of u 0 
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such that FCu> <z V for every u € V. F is said to be upper 
semicontinuous if it is upper semi-contonuous at every point u e U . 

The correspondence F is called usco, if it is upper semi-continuous 
and FCu) is non-empty and compact for every u e U . 

2. 2. Definition. A multivalued mapping F : U 4 > U 2 from a 

topological space U 1 into a topological space U 2 is said to have the 
property at u e U 1 if there exists a point w in FCu> so that 

for each neighbourhood V of w there exists a neighbourhood V of 
u such that for every v e V the intersection FCv) n V is 

non-empty . 

In an implicit form this notion was used in CDIC3. Explicitely it 
was involved in (Chi and (ChK3 where it was shown that for each usco 
mapping F : U l > U 2 from the complete metric space U 4 into the 

Eberlein or Radon-Nikodym C see CN3 ) compact space U 2 there exist a 
lot of points C forming a dense and G^ subset ) of U t at which F has 

the property C*>. In C1C33 this is also proved for usco mappings F 
with Gul'ko compact range space U 2 . 

2. 3. Definition. A Hausdorff compact space X is said to belong to the 

class S if for every complete metric space B and for every usco 

mapping F : B > X there exists adense subset G of B such 

that for every b e G the mapping F has the property <*> at b. 

The class S has remarkable stability properties : it is closed 
under taking countable products, countable sums of closed sets and 

perfect images. 

The idea to consider the class of all spaces X for which the 
property discussed in definition 2.3 is fulfilled belongs to Stegall 
tS13. He considered the C eventually smaller > class of spaces X 

obtained if one replaces in definition 2.3 the words "complete metric" 
by "Baire". The properties of the class 8 and of the spaces in it are 
discussed in CS1 ,S2 ,S3,S4,K1 ,K2,8K1,&K2 3 . Ve would like to mention here 
that C contains C in increasing generality > all Eberlein compacta, 
all Talagrand compacta [D3, all Gul'ko compacta CK33 and all 
fragmentable spaces CR3. 
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3* "Max Min" problems. 



3.1. Theorem. Let X and Y be Hausdorff compacta and X x Y belongs 

to 32. Then the set of all functions f <e CCX x Y> , for which the 
corresponding "max min" problem C P^ > has unique solution, contains 

a dense subset of the space CCX x Y) equipped with the usual 

uniform convergence norm. 

Proof. 



Let us consider the multivalued mapping S : CCX x Y) > X x Y 

assigning to each continuous function f e CCX x Y) the set of 
solutions of C P^ ), i.e. 



SCf) 



f Cx o> y 0 5=n>ax<f <x Q , y> : yeY>= 
< <x y ) « X x Y : ° ° ° 

° max< minCf Cx, y) : yeY>: xeX> 



First we will 
set-valued mapping. 


show that S 
Next we show 


: CCX x Y) 
that S 


> X x Y is an 

is single-valued at 


usco 

the 


points where it has 


the property 


C*>. Since 


X x Y belongs to 32, 


this 



will complete the proof. 



3.2. Lemma. 



is usco . 



Proof of the lemma. 



Since X x Y is compact, it suffices to prove that the graph of 
S is a closed subset of CCX x Y> x CX x Y). 

Ve will show first that 
Cl> < min < 



,<x 0 ,y> : yeY > 




— > min < f (x_ 
o o 


II. II 


< r a > aeA 


-> f o 


and < x a > aeA - 



-> x o- 



Indeed, let min < 


f a <x et’ y:> : y « Y > 


= f a <x a’ y a 5 


for some 


ya 


e Y and min < f Q Cx 0 ,y>: 


y e Y > = f <x , y ) 
J o o ' J o 


for some 


y o e Y - 


The 


compactness of Y yields a convergent 


subnet < 




of 


< y a > aeA - Let y be its 


limit. Then 








C2> f a Cx Q ,y Q ) > min < 

ft ft 


f a <x a ’ y) : y e Y 

ft ft 


V 

II 

rs 

X 

< 


V 





But 
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i f a„ <x «y - f o <x o> y > i * 

l f « J>-^ C *a^ y a >1 + I f n Cx „ > Y„ >- r a Cx a’ y5 I 5 



°ft a ft a f) 



ft "ft 



*ft "ft 



" % - f o 11 + I I 

and hence { l* a <x a ^,y a ^) > ^ > f 0 <x 0 ,y) . Analogously 

< f Cx^ , y_) p > From the inequality C2) we obtain 

C O ( 5€JD O O O 

f Cx rt ,y rt ) > f Cx_ , y ) , 
o o’-'o o o 7J • 

On the other hand 

fo^o’S* 2 min * f o <x o’ y:> : y e Y > = fo { *o-V- 
We conclude that 

V x o' y> = min { f o Cx o' y> : y e Y > 

and so 

{ min < f a Cx ,y): yeY >1^ > min < f 0 <x 0 ,y>: yeY > 

Ve obtained that every subnet of min < ^ a (x a >y ): y eY > has a 
subsubnet converging to min < f C x Q ,y> : y e Y >. Hence Cl) holds. 

Let < C f a , x a , y a > > ae A is a covergent net contained in the 
graph of S GrCS) C i.e. Cx a >y a ^ « SCf a ) for ever y cx e A ). Let us 
denote its limit by ^ 0 ,x o ,y o^' Then 

f a Cx a' y a >ss min < f a Cx a’ y:>: yeY y “ min * f a Cx ' y:>: yeY y 

for every a <s A and every x e X. Using Cl) we conclude that 

f Cx , y )= min < f~Cx_,y): y <s Y > > min < f Cx,y): y <s Y > 

O 0*^0 O O 7 J O ' "* 

for every x <= X, i.e. (x 0 ,y 0 ) e SCf Q ) or Cf 0> x 0 > y o^ € GrCS). 

Therefore GrCS) is a closed subset of CCX x Y) x CX x Y) and, 

because of the compactness of the range space X x Y, S is upper 
semi-continuous. It is evident that S has non-empty compact images 
because of Cl).« 



3. 3. Lemma. Let S have the property 


C*) at 




CCX x Y) and 


Cx o> 


y o> 


be the point mentioned 


in C*), i.e. 


for every 


neighbourhood 


U 


of 


Cx 0 ,y 0 > there exists 


e > 0 so that the 


intersection SCf) 


n U 


is 


nonempty whenever f e 


CCX x Y) and 


II f - 


f o 11 


< e. Then 


the 


set 



SCf Q ) has only one point, i.e. SCf Q ) = < Cx 0 ,y Q ) >* 
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Proof of the lemma. 

Let us suppose that SCf Q > contains more than one element. Ve 
will consider two cases. 

Case 1. There exists a point ^x ± ,y ± > e SCf Q > such that x Q * x ± * 

Then there exits a continuous function a defined on X such that 
II a N CCX )“ aCx 4 > £ 0 and aCx 0 > > 0. Let us denote by V the open 

set < x g X : aCx> >0>xYc X x Y t For every e > 0 we consider 

the function f £ e CCX x Y> defined by f £ Cx,y) ® f Q Cx,y) - e.aCx>. 

It is clear that II f - f II < e. Ve will show that SCfO n V * 0 

O C fc- 

which will contradict the property C#0. 

Let <x,y) be an arbitrary point of V. Ve have 
min < f c Cx,y) : y e Y > * 

min < f Q Cx,y) : y e Y > - aCx) £ v Q - e.aCx) < v Q , 
where v Q = max < min < f Q Cx,y) : y e Y > : x e X >. 

On the other hand, since Cx 1 ,y ± ) e S(f Q >, we have 



min 


< 


r o Cx 1 ,yy : 


^ y 


€ Y 


> 5 


min 


< 


f e <x 1>y > : 


y 


€ Y 


V 

IA 



max < min < t e Cx,y'> : y e Y > : x e X >, 

Therefore 

min < f^CXjy) : y e Y > < v Q < 

max < min < f e <x,y> : y«sY>: x <= X > 
and hence <x,y) ^ SCf^). 

Case 2. There exists a point y ± e Y, y % * y Q> for which Cx Q ,y e 

SCf ). Hence there exists a continuous function b on Y with 

o 

II b H CCY3 * 1> b<y 1 > < 0 and b(y Q > > 0. Ve denote by V the open set 

X x < y e Y : bCy) > 0 > c X x Y. 

Let e > 0 be an arbitrary positive number. Consider the 

function X x Y > R defined by 

' f Q Cx,y) + e/5 if f Q Cx,y) < v Q - c/5 

ff £ Cx,y) « v Q if f Q Cx,y) «e i v Q - c/5, v Q + e/5 3 

f 0 Cx,y> - c/5 if f 0 Cx,y) > v Q + c/5 

where v D * max < min < f Q Cx,y) : y e Y > : x e X >. 
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It is streightforward to check that is continuous on X x Y ? 

tl s e " 11 ^ and 

v Q * min < ^ s <ix o> yy 1 y € Y > = 





max 


< min < g £ Cx,y) : 


y 


e Y 


> : 


x e X >. 


The compactness 


of 


Y and f 0 Cx o' y:> 


> 


v o" 


e/5 


for every y e 


Y yield the existence 


of 


a neighbourhood 






of 


x Q for which 



f Q Cx > y> > v Q - e/5 for every x e and for every ye Y. Let 

be a neighbourhood of x Q for which f Q Cx,y 1 ) < v Q + e/5 whenever x e 

Ur. We set U = U' n U'' . Thus U_ is a neighbourhood of x with 

£ £ £ £ & ^ 

g^Cx,y 1 > = v Q < g £ Cx,y) for every x e U £ and for every y e Y. 

Consider the function f £ : X x Y > IR defined by 

f £ (x,y) = g £ Cx,y) + e/5.bCy> + 3e/5.a e <x) 

where a £ is a continuous function on X with a e Cx Q ) * 1, 

a £ Cx) * 0 for every x « U £ and I! a g l! CCX) = 1. 

It is evident that f £ is continuous and 
II f Q - f £ II £ e/5 + e/5 + 3e/5 = e. 

¥e will show that SCf^) n V = 0 for every e > 0. This will 

contradict the property (*). 

Let Cx,y) be a point in V. If x belongs to we have 

g £ Cx,y 1 ) < s s Cx,y> by the construction of U e and hence f c <x,y t > < 

f e (x,y), because bCy ± ) < bCy) for Cx,y) in V* Then 

f £ Cx,y) * min < f £ Cx,y) : y e Y > 

and so <x,y) doesn't belong to SCf^.). 

If x U £ , then a e <x> = 0 and we have 

min < f Cx,y) : y e Y > = 

min < g £ Cx,y) + e/5.bCy> : y e Y > < 

min < 6 e Cx,y) : y e Y > + e/5 < 

v + e/5, 
o ’ 

On the other hand 

min < f e Cx Q ,y) : y e Y > * 




47 



min < 6 c ^x 0> y) + &/ 5.bCy) : y e Y > + 3s/5 £ 

min < g c <x ,y> : y <e Y > - c/5 + 3c/5 = 

v 4- 2c/5 t 
o 

Therefore 

min < f e Cx,y) : y e Y > £ v Q + e/5 < 

v Q 4 2e/5 £ min < f £ Cx 0 ,y) : y e Y > 
and hence Cx,y) doesn't belong to SCf c ).« 
Theorem 3.1 is proved. 
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Optimal stopping and leavable gambling models with observation costs. 



Manfred Schal, Universitat Bonn, Institut fur Angewandte Mathematik, 
Wegelerstr. 6, D-5300 Bonn 1. 



Summary . It is well-known that for the problem of stopping a Markov 
chain with finite state space there exists an optimal a.s. finite 
stopping time which is the entrance time into the set where the value 
function coincides with the utility function. In this paper, this 
result is extended to the case where more than one continue action 
is available. The result of the paper also yields a sufficient condi- 
tion for the existence of a stationary optimal policy in an extended 
leavable gambling house with compact action space where observation 
costs are incurred. 



Key words and phrases: Dynamic programming, gambling, optimal stopping, 
average return, limit supremum payoff, stationary policy, finite 
stopping times. 
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§ 1 Introduction. 

We consider a system with a finite number of states i € S. Periodically 

we observe the current state of the system and then choose an action 

a from a set A of possible actions. As a result of the current 

state i and the chosen action a , the system moves to a new state 

j with probability p^ ^ (a) . As a further consequence, an immediate 

cost c(i,a) is incurred. Our desires are summerized, in a certain sense, 

by a function u from S to real numbers, called the utility function. 

The worth of the state i is u(i) . If h = ( i Q , i^ , i 2 , • • • ) is a possible 

history of the system which stagnates at time n, i.e., 

i =i 1 „=i l0 =... , then the history describes the case where no 

n n+ 1 n+ 2 

observations are made after time n . Such a history is valued accor- 
ding to the utility u (i R ) of the terminal state minus the observation 
costs {c (i Q , a Q ) + ... + c (i n-1 , a n _^ ) } where a fc is the action chosen 
at time t . 

If the history does not stagnate at some finite epoch n, then it is 
no longer obvious how to value the history. In the framework of gambling 
models (cp. Dubins & Savage (1965), Sudderth (1971)) and optimal stop- 
ping models (Chow, Robbins & Siegmund (1971)), the history is valued 
according to 

(ni) (n(i n )-z":J c(i t ,a t )}, 

where however c = o in the gambling literature. Actually, for an infi- 
nite state space which is not considered here, this definition is 
essential in order to get optimal possibly infinite stopping times for 
stopping problems. Then a policy which specifies how to choose the 
actions is valued according to the expectation of (1.1). 

In dynamic programming with the total cost criterion, the optimality 
criterion for policies is given in a natural way. There u=o and 
£~_ o c(i^,a^) exists; moreover the sum and the expectation can be per- 
muted by virtue of the usual convergence assumptions. However, in the 
case of the present paper, these assumptions are not satisfied mainly 
because u(i n ) will generally not converges for non-stagnating histo- 
ries. Therefore we will borrow methods from dynamic programming with 
the average cost criterion which also deals with non-convergent sequen- 
ces. There one make use of the more general concept of the limit in 
the sense of Cesaro and Abel. 

The present paper has predecessors. In Schal (1985), the case c=o is 
treated. There, results for the average cost criterion directly apply. 

In Schal (1986) the present situation with observation cost is treated 
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for a Cesaro criterion and an Abel criterion. The purpose of the pre- 
sent paper is to strenghthen the latter results by extending them to 
the limit supremum criterion (1.1). There we can built on results by 
Schal (1986) and on methods for gambling models with observation cost 
(or equivalently: with intermediate one-step rewards) presented in 
Schal (1986a). 

§ 2 The model 

The underlying model is given by a tupel (S,A,p,c,u) . 

(i) S stands for the state space and is assumed to be finite. 

(ii) A is the action space . 

(iii) p i .(a) are the transition probabilities , where P i j(a) >_0 and 

Y . p. .(a) =1 for all i€S,a€A. 

L }€S 

(iv) c(i,a) is the real valued cost function . 

(v) u(i) is the real valued utility function . 

The results will easily extend to the case where the set A of available 
actions depends on the current state. 

We write H t = S*...xs (t+1 factors) for the set of all histories 
(i , ...,i fc ) at stage t . A policy 6 = (6 fc ) is defined as a sequence of 
decision functions 6^ : A. We write A for the set of all poli- 

cies. An important role plays the set F of stationary policies. They 
can be identified with decision functions f : S -► A where 
6 = f iff 6 t (...,i t ) = f(i t ) for all t. Given the initial state i , 
any policy 6€A defines a stochastic process (X ) where X n describes 
the state of the system at stage n. The process (X ) is a Markov chain 
if the system is controlled by a stationary policy f€F . Let us write 
= (X Q ,.../X n ) for the history up to stage n and 

(2.1) u n <6) = u(X n ) - i*Zl c(X t ,6 t (X (fc) )) 

for the gain if we stop at stage n . 

The cost function may be negative and is then to be interpreted as 
reward. However, we assume that the total expected cost is bounded from 
below in the following sense: 

Assumption (B) : SU P6€A E i6 ^t=o C ^ X t ' ^ t ( t) ^ ^ <0 ° for a ^ 1 

This is the weakest assumption which is made in dynamic programming 
with the total cost criterion where u=o , and (B) defines then the 
so-called quasi-convergent case. However, our model is not a special 
case of this quasi-convergent case because of the utility function u 
and needs more general methods. 

For any stopping time t with respect to the process (X n ) which is 
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P i( $ - a.s. finite, i.e., P i( $ ( T<0 ° ) <0 ° , in particular for a constant x=n 
the expected gain up to stage x is now well-defined as 

(2.2) v l6 (i) = E i6 [u T (S)], 
since u is bounded and 



(2.3) u n (<S) £ SU P u + ^t=o C ^ X t' 5 t^ X (t) ^ ’ 

If x is not finite, e.g. if x = 00 everywhere, the definition of the 
expected gain is more problematic. For this paper we will use the 
following definition which essentially stems from the theory of 
gambling: 



(2.4) v 5 (i) 



E i6 [lir Voo V 5)] 



Again, by (B) , is well-defined. Now, the value function is 



(2.5) v ( i) = sup 6eA v 6 (i) . 



Remark . Assumption (B) may even be replaced by the following weaker 
assumptions: v^ is well-defined, v<°° , {(u n (<$)) + ) is uniformly 

P^-integrable for all i€S,6€A, {(u n (6)) } is uniformly P^-inte- 

grable for all i€S,<5€A , such that v^(i) > -°° . □ 

We want to incorporate stopping into the policy. Therefore we assume 
that the underlying model is leavable, i.e. we make the following 
general assumption. 

Assumption (A) : A = A q + {e} where p^ (e) = 6^ and 

c(i,e) = 0 for all i, j 6 S . 

Choosing action e means that one pause for one period, i.e., the 
system remains in the same state and no cost is incurred. As Dubins & 
Savage (1965), we interpret stopping at time x by continuation such 
that one always chooses action e from stage x onwards. Then, with 
probability one, the system remains in the state of stage x time 
after time and u^(6) = Tim u n (6) . 

2 . 6 Lemma . Let 6 € A and x be a stopping time such that 
P i6 (x <0 °) = 1 and 6 t (X^) = e for t^x, then v^(i) = v T ^(i). 

In the case where A q is a singleton set, one has an ordinary stopping 
model . 

2 . 7 Lemma . u < v < 00 . 



Proof . For the stationary policy f where f(i) = e for all i€S one 
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has v f = u , hence u £ v . From (B) and (2.3) we get v < » . □ 

In addition to (2.3) we need an integrable minorant for ( 11 ^( 6 )}. Now 

(2.8) u n ( 6 ) > inf u - £“ =o c + (X t , 6 t (X (t) )) . 

Integrability follows from 

2 . 9 Lemma . c + (X fc , 6 1 (X ^ ) ) ] < 00 if v^ (i) > - 00 . 

Proof. Let us write = C(X t , 6 t (X (t) )} * Then 

Itli < = l£, c t + u(X n ) - u n (6) • Hence ' 

ri OO -f- „00 — 

). < ) ^ Cj_ + sup u - lim u ( 6 ) 

^t=o t — L t=o t * n 

by taking lim on both sides. Now (B) applies. □ 



§ 3 Relation to the Cesaro-Abel-criterion. 

Another definition of the gain can be based on the limits in the sense 
of Cesaro and Abel. 



(3.1 ) 


c 

v <5 = 


lin V°o i (v u 




■ + v nS> 


(3.2) 


A 

v 6 = 


I “ 6t 1 (1 ~ e) ‘ 


-.OO 

^n=o 


6 n v n6 , 6 £ A . 


3 . 3 Lemma . 


v i 1 v 5^ v 6 


for 


6 € A. 


Proof . 


By 


(B) and (2.3) , 


the 


sequences {v ^(i)} is bounded from above 



Now the first inequality follows from a well-known relation between 
the Cesaro and the Abel limits (cp. Hordijk (71)) . Again by (B) and 

(2.3) , F^tou's Lemma applies and one obtains 

(3.4) lim v n< 5 (i) £ Vg(i) , i€S. 

Now the second inequality is obvious. □ 

From the theory of Markov chains one knows that the limits in 

(3.1), (3.2) exist and coincide for stationary policies f € F . 

Therefore we write 

(3.5) w f = v f = v f ' f € F . 

(3.6) w = sup f e p w f . 

From the argument for 2.7 and from 3.3 we obtain 

(3.7) u £ w £ v < 00 . 

Now let JPf be the transition operator for the Markov chain under f? 
i.e. for any real-valued function r on S one has 
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(3.8) 3P f r(i) = ^ j£S Pij (f (i))r(j) , ies , 

and let be defined by c f (i) = c(i,f(i)) , Then clearly 

(3.9) v Qf = u , v n+1>f = 3P f v n - Cf 

and as a consequence since the limits in (3.5) exist: 

(3.10) w f = 3P f w f - c f . 

3.10 Proposition , (a) For e > 0 there exists some f € 3F such that 
> w - e . 

(b) (superharmonici ty ) 

w(i) 1 Ij e s P i j(a)w(j) - c(i,a) , i£S , a£A . 



Proof . a) By definition, one has for e>0 and i € S some f ^ 6 F 

such that w. (i) >w(i) - e. Now consider a model where A is 
f i ° 

replaced by { f ^ ( j ) ; i, j 6 S) = A q . The policies f i are admissible 

for this submodel. By the finiteness of A q we know from the results 

of Schal (1986) that there exists an optimal stationary f for the 

submodel. Hence w^(i) ^ w^ (i) >_ w(i) - e . 



b) Let be e>0, f as in part (a) , and cp E F arbitrary. Now consider a 
submodel where A q is replaced by {f (i) ,’ <p(i) , i € S} . For this 
submodel the non-stationary policy 6 = (cp,f,f,...) is admissible. For 

6 , similarly to (3.5) and (3.10), the limits in ( 3 . 1 ) , ( 3 . 2) exist and 

C A 

coincide and one has v <$ = v <s = ^ w f ” . Let f Q be optimal for the 

submodel. Then 



w > w f > Vr. = F w^ - c > IP w 
— f 0 — <5 cpf cp — cp 



C <P - £ 



Now define similarly to (2.1): 



(3.11) w n <«) = w(X n ) - I^c(X t ,6 t (X (t)) ). 

3.12 Lemma . Let be 6 € A , i € S such that v P ( i) > - 00 . Then {w ( 6 ) } is 
o n 

a supermartingale, i.e., 



E 6 Cw n+1 (6) l X (n) - 1 1 w n (6) for all n . 
Moreover, lim w n (6) exists and is P^-integrable where 

E. ~ lim w ( 6 ) < w(i) . 

10 n — 



Proof . A relation similar to (2.3) shows that (w n (6)) + is integrable. 

Since w>u and Vr(i) >-», w (6) > u (6) and {w (6)} has a 
— o n — n n 

P i($ -integrable minorant by (2.8) and 2.9. Now for h = (...,i) € 
E 6 [w(X n+ i) - c(X n ,6 n (X (n) ))|x (n) 



= h] 
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= ^j€S p ij (6 n (h) )w(j) " c < i ' 6 n (h)) 

- w (i) by 3.10b 

which shows the supermartingale property. 

By the martingale convergence theorem, lim w (6) exists a.s. and is 
P^-integrable . By Fatou's Lemma 

E . * lim w (6) < lim E. ~ w (6) < E . x w (6) 
io n — i0 n — lO o 

since {E. x w (6)} is decreasing. Finally w (6) = w(X ). □ 
io n ■* o o 

3.13 Theorem. w = v . 

Proof . Let be i 6 S, 6 € A such that v^ (i) > - °° . Then by 3.12 

w(i) > E.o lim w (6) > E . « lim u (6) = v Ji) 

— io n — io n o 

since w _> u. Hence w >_ v . But w _< v is known from (3.7) . □ 

3.14 Corollary. Let be f € 3F . If w f = w , then w f = v f = v . 

Proof . From 3.3 and 3.13 one knows that 
w=w^£v f £v = w. □ 

S 4 Optimal policies. 

In order to obtain optimal policies we assume compactness and conti- 
nuity assumptions. 

Assumption (C) : (1) A q is a compact metric space and e is an 

isolated point. 

(2) a -*■ p^j (a) is continuous for all i,j. 

(3) a ■+ c(i,a) is lower semi-continuous for all i. 

As in Schal (85) , (86) we need an additional continuity assumption. We 

premise the following lemma which is easy to prove under (C) . 

3 . 1 Lemma . The following conditions are equivalent for all i,j.: 

(i) {a € A q ; p i ^ (a) > o} is closed ; 

(ii) a -► sign p^ ^ (a) is is continuous on A q ; 

( iii) inf (p^ta) ; a£A Q , p i j(a)>o}>o . 

Assumption (C*) ; For all i,j€S one of the conditions in Lemma 2.1 
is satisfied. 

Two important examples where (C*) is satisfied are the cases where 

A q is finite or where (for all i,j€S) p^fa) = 0 either for all 

a € A or for none . 
o 
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On combining Corollary 3.14 with results from Schal (1986, Theorem 2.4, 
Corollary 2.5) one obtains: 

3.2 Theorem. There exists an optimal stationary policy f, 

i.e., v f = v, such that 

f (i) = e if v(i) = u(i) 

f (i) € A q if v ( i ) * u(i) . 

Moreover = v ^ where t is the entrance time into the set 

[v = u] and E if x < °o for all i€S. 

In Schal (1986) continuity of a c(i,a) was assumed rather than 
semi-continuity. However the proof given there can be extended to the 
case of the present paper (cp. Schal (1986a, §17)). 
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Abstract 

Nonlinear programing problems are considered vtfiere the objective function in- 
volves several ratios. We review recent results concerning three classes of mul ti - 
ratio fractional programs: 1) maximization of a sum of ratios, 2) maximization of the 

smallest of several ratios, and 3) multiobjective fractional programs. In addition to 
these results open problems are addressed as well. 

Introduction 



The large majority of articles in fractional programming deals with the optimi- 
zation of a single ratio. Until 1980 only a few isolated results had appeared on 
multi-ratio fractional programs. Interestingly enough one of the earliest publications 
contributing to fractional progranming, J. von Neumann's classical paper on a model of 
a general economic equilibrium in 1937 [26, 27], contains an analysis of a multi-ratio 
fractional program. In this early paper even a duality theory for this nonconvex pro- 
gram had been suggested, at a time when linear progranming was hardly known. 

Even at the recent NATO Advanced Study Institute on "Generalized Concavity in 
Optimization and Economics" [24] in 1980 most papers in fractional programming con- 
tributed to single-ratio fractional progranming. However, since that time a good 
number of results have appeared in multi-ratio fractional programming. A new area of 
research has been opened up. 

Most of the contributions deal with one of the following types of multi-ratio 
fractional programs: 1) maximization of a sum of ratios, 2) maximization of the small- 

est of several ratios, and 3) multibbjective fractional programs. In this survey we 
report on results for each of these classes of fractional programs. We mainly focus on 
results in the derivation of which the author has been personally involved. 

1. Maximizing a Sum of Ratios 

Consider 

P fjx) 

(1.1) l — -*• Sup 

i=i g i (x) 

subject to 

X £ S 

where S -R n is a convex feasible region. In single-ratio fractional programming 
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the classical assunption often met in applications is that the numerator is nonnega- 
tive and concave and the denominator is positive and convex. Then a local maximum is 
always a global raximum, duality relations can be established and several algorithms 
are available to solve such concave-convex single-ratio fractional programs [19]. Un- 
fortunately, for nulti-ratio fractional programs (1.1) none of these properties is true 
any longer if the classical assunption on f^/g^ is satisfied [18]. In particular, a 
local maximum may net be a global one, even if a simple function like the sum of a 
linear ratio and a linear function is considered. Almost no results are known yet 
about the properties of (1.1) and how to solve this problem efficiently if f^/g^ are 
concave-convex . 

On the other land, some limited progress oould be made for the maximization of 
the sum of certain non-concave-convex ratios and a concave or convex function [21]. 

Such a problem arises when the sum of a relative and absolute term is to be maximized. 
It turns out that the objective function is often quasiconcave or quasiconvex [2] if 
f i^ g i °°ncave-convex but satisfies other kinds of convexity assumptions [21]. 

Another example of model (1.1 ) which is tractable and where the classical assump- 
tions on f i /g i are not satisfied is discussed in [23]. The model analyzed in this 
paper is designed to determine simultaneously optimal lot sizes and an optimal storage 
allocation in a warehouse. The total cost to be minimized is 



( 1 . 2 ) 



K(x) 



= I 

j=i 






x . 
3 



+ B .x. 

j 3 J 



l 

j=i 



yJ( i 

J m= 



mFl 



,) s/i 



j-l 

( I 

m=l 



,) s/j 






Here the first term is the fixed cost per unit, the second one the storage cost per 
unit and the last one the material handling cost per unit. In the last expression the 
ratios are not convex-concave, in fact they are not even quasiconvex. However, it can 
be shown that there exists a cne-to-one continuous tranformation of variables such that 
the transformed function of K(x) is (strictly) convex. This proves that a station- 
ary point x (v K(x) =0) is a (unique) global minimum of K(x). Hence the minimum 
of K(x) can be calculated in a straightforward manner. 

We see from this application that a sum-of -ratios problem (1.1) nay well be tract- 
able if the ratios are not concave-convex . Of course, the major challenge remains to 
derive properties for (1.1) when the ratios are concave-convex. 



2. Maximizing the Smallest of Several Ratios 
Consider 

f Ax) 

(1.3) Min — -► Sup 

1 _< i £ p 9^( x ) 

subject to 

h^x) <^0 k = 1, . . ., m 
x e S 0 
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where S 0 £ R n is nonerrpty, convex 
h^: S 0 -► R are convex. 

In contrast to (1.1) problem (1.3) is known to have very useful properties in case of 
ooncave-convex ratios fj./gi« Before going into details we want to first point out 
some applications of (1.3). 

In mathematical economics problem (1.3) arises When determining the grcwthrate of 
an expanding economy [26, 27]: 

(1.4) grcwthrate = Max ( Min ? _ ^ ) 

x 1 _< i _< p inputs (x) 

Where x denotes a feasible production plan of the economy. 

In management science simultaneous maximization of rates can be achieved by 
either using a worst-case approach 




Sup 




In both cases a nax-min fractional program (1.3) is to be solved. The second ap- 
proach is used in a financial planning model by Ashton/ Atkins [1] Where liquidity, 
return on capital, dividend per share as well as other financial ratios are maximized 
simultaneously. Model (1.6) is also used in a resource allocation problem by Chames/ 
Cox/Lane [6] designed to allocate funds under equity considerations to educational 
institutions in the state of Texas. 

A third area of application of model (1.3) is numerical mathematics. Given the 
values Fi of a function F(t) in finity many points t^ of an interval for Which 
an approximating ratio of two polynomials N(t; x^) and D(t? x 2 ) with coefficient 
vectors x 1# x 2 is sought. If the best approximation is defined in the sense of the 
L^-norm, then the following problem is to be solved: 

N(t . ; x x ) 

(1.7) Max - F. -> Inf 

i D(t.y x 2 ) 



for variables x 1# x 2 [3]. Like (1.6) this problem can be reduced to the nax-min frac- 
tional program (1.3). 

Several authors including J. von Neumann [26, 27] have introduced dual programs 
for problems of the type (1.3) using different approaches? for references see [8]. 
Recently two new approaches were suggested in [8] and [14]. The first one enploys the 
framework of quasiconvex duality Whereas the second one does not leave the confines of 
convex analysis. Very recently Scott /Jefferson [25] derived similar results with help 
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of conjugate duality. 

In most duality approaches for (1,3) the following assumptions are made: 

S 0 is compact 

”fi' g^, h^ are lcwer-semucontinuous 
-f^, g^ are convex 
g^ are positive on S 0 

f^ are nonnegative on S, if at least one g^ is not affine. 

Herein S = {x e S 0 | hj^(x) _< 0 k = 1, . . . , m} denotes the feasible region vhich 
is assumed to be nonempty. 

Let F = (f x , ..., fp) T , G = (g x , . .., g^) T and h = (h 1# . .., bm) T . The 
foliating dual was derived in [14]: 

(1.8) sup - yTh(x) . inf 

x e s 0 uG(x) 

subject to 

u _> 0, u * 0, v_> 0. 

Under the assumptions above one can shew [14]: the optimal values in (1.3) and (1.8) 

are the same. Additional duality relations resembling those in convex and linear 
programming can be derived as well [8, 14]. 

We note that the primal max-min problem (1.3) is associated with a dual min-max 
problem (1.8). Such a symmetry is not obvious in single-ratio fractional program- 
ming duality theory [16]. Symmetry between the primal and dual exists also in the 
following sense: in both problems a local optimum is a global optimum. This follows 

frcm the fact that the primal objective function is semistrictly quasiconcave and the 
dual objective function is semistrictly quas icon vex [2]. 

The dual objective function usually involves infinitely many ratios in contrast 
to the primal one. However this asymmetry disappears in case of a linear problem 
(1.3) where f^, g^ and h^ are affine and S 0 is the (unbounded) nonnegative 
orthant of R n . Then only finitely many ratios need to be considered in the dual ob- 
jective function [8, 14]. In the linear case it can further be shewn that in addition 
to the usual complementary slackness between variables in one problem and constraints 
in the other one complementary slackness also exists between certain variables in one 
and ratios in the other one. For details see [8]. Summarizing, we can say that for 
linear problems (1.3) there exists complete symmetry as well as intimate relations be- 
tween the primal and the dual fractional program. 

We realize that for max-min fractional programs (1.3) with concave-convex ratios 
a rich duality theory can be established. We will now turn to an algorithm that solves 
these same fractional programs [9]. In the linear case of (1.3) the method can also be 
applied to the dual as well because of the symmetry between the primal and dual. The 
procedure is a generalization of Dinkelbach's algorithm [11] for single-ratio fraction- 
al programs. In case of one ratio only the method is theoretically as well as ccmputa- 
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tionally very appealing [17, 22]. 

Consider 

f. (x) 

(1.9) 0 = Max ( Min — ) (P) 

x e S 1 < i <p g^x) 

where S - R n is nonempty, compact 

f i , gi : S R are continuous and g^ > 0. 

We associate with (1.9) the parametric program 

(1.10) F(q) = Max Min [f. (x) - q g. (x) ] , q e R. (P ) 

xeS 1 _£ i <_ p 1 4 

F(q) is a strictly decreasing, continuous function. The idea of the algorithm is to 
find a zero q of F. It is easy to see that an optimal solution of (P-) is an opti- 
mal solution of (P), and q = 0. 

(Pq) is a convex program for q ^ 0, if f^/g^ are concave-convex ratios. (P^) 
can be written as a linear program in the linear case of (P), i.e. if f^, g^ are 
affine and S is a convex polyhedrcn. In either case the parametric problem (P^) 

(q given) is easier to solve than the original problem (P). 

The following algorithm was suggested in [9]s 

. f . (x° ) 

Step 1: Let x° e S, q, = . — , k = 1. 

1 g i (x°) 

Step 2: Solve (Pq^); let x be an optimal solution. 

Step 3: If F(q^) = 0, then step; x is an optimal solution of (P) and 

, k = k + 1, 

The sequence {q^} is increasing and converges to q. It does so linearly. 
Furthermore, each convergent subsequence of {x^} converges to an optimal solution of 
(P). For details see [9]. 

It is interesting to contrast the algorithm by Dinkelbach [11] for p = 1 with 
its generalization for p > 1 in [9]. For p = 1 we have: 

a) F is convex on R, 

b) the algorithm is equivalent to Newton's algorithm, i.e. it generates the same 
sequence {q^}, and 

c) the algorithm converges super linearly. 

For p > 1 none of these properties hold anymore. In particular, the algorithm is 
only linearly convergent in general. On the other hand, the procedure has two advan- 
tages over Newton's method: 



If F(<^) * 0, let 
and go back to Step 2. 



. f . (x k ) 
Min l 

i , kv 
) 
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1) F need rot be convex, and 

2) a subgradient of F need not be known. 

The disadvantage of slew convergence can be overcome in cases when (P) has a unique 
optimal solution. This is achieved by solving a somewhat modified subproblem in Step 
2 [10, 13]: 

(1.11) Max ( Min [ f^x) -q^g^x)]) . 

x e S 1 < i £ p g i* x * 



It can be shewn that {q^} is increasing and converging to q. It does so linear- 
ly. However, if the optimal solution of (P) is unique, then {q^} converges even 
superlinearly and {x^} converges to an optimal solution of (P). For details see [10]. 
Under more restrictive assumptions on (P) the convergence rate can be made 1.6 or even 
quadratic [5, 13]. 

The computational experience with the original algorithm in [9] and its modifica- 
tion in [10] is presented in [12]. The modified algorithm proves to be clearly super- 
ior to the original one. 

So far the feasible region of (P) was assumed to be bounded. In the unbounded 
case F(q) may no longer be finite and then the algorithm will fail to work. How- 
ever, it can be shewn [9] that at least in the linear case of (P) the dual as intro- 
duced earlier will always have a finite F- function. The algorithm can therefore be 
applied to the dual. Hence the problem of an unbounded feasible region of (P) can be 
overcome by solving the dual instead of the primal, at least in the linear case of (P) 



3. Multiobjective Fractional Program 



Consider the general nultiobjective program 

(1.12) Max ( F^x), ..., F (x)) , p 2 
x e S p 

where s£R n is nonempty, compact, 

F-l : S -► R are continuous. 

Let E be the set of efficient (Pareto-optimal) solutions and F(E) the efficient 
frontier. 

Bitran/Magnanti [4] shewed that E is a connected set, if S is convex and all 
F^ are concave functions. We recall that a set M is called connected if there do 
not exist closed sets Qjl , Q 2 , such that M £ Qjy Q 2 t M n * 0 , M r\ Q 2 * 0 and 
Mr\(Q 1 n Q 2 ) = 0 . 

The question arises for which nonconcave functions Fi the set E is still 
connected. In particular, will it be connected for nultiobjective fractional programs 
and for which kind? Unfortunately, it does not seem to be possible to carry over the 
analysis by Bitran/Magnanti to nonconvex nultiobjective programs since the authors use 
the parametrization £ X^F^(x) to generate E. 
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In case of a single objective function (p = 1) E is identical with the set of 
optimal solutions. This set is still convex, and thus connected, if F(x) is quasi- 
concave [2]. 

In the bicriteria case (p = 2) quasiconcavity of F x , F 2 does not always guar- 
antee connectedness of E as the following example shows: 



S 



[-1, 1], F x (x) = x, F 2 (x) 



^-x -1 < x < 0 

^ 0 0 _< x _< 1 



Here E = [-1, 0)U {1} . This example also demonstrates why E is disconnected? name- 
ly because of the flat portion of F 2 (x). On the other hand, if F 1# F 2 are both 
semistrictly quasiconcave [2] (and not just quasiconcave), then E can be shown to be 
connected [20]. The proof makes use of the parametrization 



(1.13) Max F 2 (x) , Ell a l F l • 
x e S 
Fi(x) >_ a 

E is the union of the sets of optimal solutions of (1.13) each of which is convex and 
therefore connected. 

Now let us consider the special case of a bicriteria fractional program 



(1.14) Max ( Mil , Mil ) . 
x e S 9l (x) 92 (X) 



If f^/g^ are concave-convex, then these functions are semistrictly quasiconcave [2]. 
Hence E is connected for bicriteria fractional programs where both ratios satisfy the 
classical assumption. E can be calculated by solving a one-parametric sequence of 
single-ratio fractional programs. 

(1.15) Max Mil . 

x e S MX) 



9l (x) 

As shown in [15, 16], (1.15) can be reduced to a convex program with help of a vari- 
able-transformation. The bounds Ej , F^ are calculated by solving two additional 
convex programs. Hence the set E of a bicriteria fractional program with concave- 
convex ratios can be determined with help of convex programming alone [20]. 

Once E is known, the "hard" problem in Section 1 



(1.16) 



Max ( Mil + Mil ) 
x e S 9 1 (X) g 2 (x) 



can be solved by finding an optimal solution of 

Max ( Mil + Mil ) 
x e E 9i(x) g 2 (x> 



(1.17) 
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with a reduced feasible region E instead of S. This is so since one of the optimal 
solutions of (1.16) is an efficient solution of (1.14). 

In case of three criteria the following result was recently derived [7]: If F 1# 
F 2 , F 3 are semistrictly quasiconcave, then the closure of F(E) (cl F(E)) is con- 
nected. The proof uses the parametrization 

(1.18) Max (F 2 (x), F 3 (x) ) 
x e S 

F x (x) _> a 

and takes advantage of the connectedness of the set of efficient solutions in the bi- 
criteria case. As a special case the nultiobjective fractional program 



fj(x) , f 2 (x) , f 3 (x) 

9l(x) ' g 2 (x) ' g 3 (x) 



(1.19) Max 
x £ S 

with concave-convex ratios f^/g^ may be considered. 

While F(E) is closed for p = 2, this is not true for p = 3 anymore. Hence 
cl F(E) * F(E). However if it is true that not only cl F(E) but even F(E) itself 
is connected, then also E is connected because of quasiconcavity of Fj_. 

At the present time two open questions remain: 

1. Is even F(E) (and therefore E) connected for p = 3? 

2. Is E connected for p > 3? 

Based on the experience with p _< 3 semistrict quasiconcavity of Fj_ has to be 
assumed when answering these questions. This included the case of concave-convex 
ratios . 
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ABSIBASI 

Vector and parallel processing are advanced computation techniques which allow to realistically 
solve a number of problems traditionally intractable. 

In this paper a short review of some basic concepts of parallel computing is given; in addition, 
motivations are provided to use parallel algorithms for solving very difficult optimization problems; 
lastly, the main progress obtained in the parallel nonlinear optimization methods is roughly sketched. 

A large scale problem of great practical interest, related to the production, transmission and 
distribution of electric energy in a network has, then, been taken into consideration and the optimal 
management policy for the system has been formulated as a nonlinear mathematical program. Its 
solution by a sequential implementation of a gradient-type algorithm exhibits a very low efficiency; 
thus, the vectorized implementation of a variable metric algorithm is considered here. The numerical 
results show its remarkable effectiveness and speed; therefore, the evidence is provided that the 
optimal operation of a power electricity network, even very large, can be successfully computed by a 
vector supercomputer when a sound and appropriate nonlinear optimization algorithm is used. 

1. INTRODUCTION 

One of the main difficulties in dealing with large scale, complex computational problems, depends 
on the use of conventional (serial) computers, which limit -often in a severe way - an effective and 
fast solution. In very recent years the availability of vector and parallel computers (e.g. CRAY I, 
CRAY X MP, CRAY 2; CDC CYBER 205, ICL DAP, Distributed Systems, etc.) has made possible, for 
their own particular architecture, the effective solution of many difficult problems. 

In the very near future, the advent of new parallel machines, at present available as prototypes, 
and in particular of compact minisupercomputers, will allow a very general application and an "at 
hand" availability of them for complex numerical computations related to mathematical problems and 
models in many theoretical and applied fields. 

In this paper some basic concepts of vector and parallel computing are preliminarly reviewed; 
then, a motivation is given - together with some references- for applying the new advanced 
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computation techniques to the solution of large scale nonlinear optimization problems. In particular, 
the problem of optimal management of an electric power network, under the respect of its economic 
operation, has been taken into consideration. 

An electric power network is a complex connection of several elements (power generators, 
trasformers, transmission lines, shunt capacitors and inductors, loads, etc.), whose fundamental goal 
is that of generating, transmitting and distributing electric energy, such that the demand for electric 
energy is fully satisfied. 

Since this important aim, optimization problems need, very often, to be solved for properly 
planning and managing the electric power systems, in such a way that the electric energy be supplied 
in the most economic, reliable and secure way. 

These problems, called Optimal Power Flow problems (OPF), are typically formulated as 
nonlinear programming problems, involving both equality and inequality constraints, and are 
characterized by a large number of variables and constraints, so that they are very hard to be solved 
by using classical algorithms. 

Although many of them have been proposed and applied, nevertheless the development of new 
algorithms particularly suitable for large scale systems, i. e. characterized by increased efficiency 
and robustness, is of very crucial importance and still an open objective of research .Therefore the 
use of parallel processing in the solution of OPF problems appears attractive and desirable, 
providing a non conventional powerful tool for overcoming the inherent limitations of sequential 
algorithms and thus for gaining extrahigh efficiency; in addition it is now widely possible, without 
severe difficulties to have access at least to vector supercomputers. 

A gradient type algorithm, namely the version of the Sequential Gradient Restoration Algorithm 
(SGRA) recently proposed by Miele, Sims, Basapur and Heideman [1] , was firstly used, in its 
sequential implementation, for the solution of OPF problems; recently it was also implemented on a 
vector computer CDC Cyber 205, and numerically tested on the same problems [2]. 

The numerical results, show that the SGRA algorithm, if properly used, may be useful in dealing 
with the solution of OPF problems, for many practical aspects. 

Nevertheless, a different algorithm based on a sound variable metric method is proposed in this 
paper; it seems preferable not only theoretically but also computationally. 

Finally, the deep superiority - in terms of efficiency - of the parallel implementation versus the 
sequential one, is pointed out and some particular features are commented. 

2. VE CTOR A ND . PARAL LEL C OM P UTING 

The development of automatic computation, by using computing systems, has always been 
characterized by limitations and difficulties. Among these one of the most important is the computing 
speed, that is to say the capacity of the system to solve, in a reasonable time and in a reliable way, a 
very complex problem. 

Then, mostly with respect to the solution of large scale scientific problems, computing systems 
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characterized by powerful hardware and software resources have always been required. 

In the past years, the increase of computing speed has been obtained by more and more 
sophisticated electronic technology. But nowadays it is really difficult to design new and more 
powerful computing systems based only on technological improvements; to increase the performance 
it is necessary to take into account the overall computer architecture, i. e. to design new organization 
models of computing systems. 

At present the information processing is dominated by the "sequential monoprocessing" methods, 
whose peculiar characteristics depend, directly, on the architectural organization of Von Neumann 
computer (the so called "control flow machine"). 

In fact, in a conventional computer, the operations (e. g. memory fetch or store, arithmetic or 
logical operations, input or output operations) can be executed only in a sequential fashion, according 
to the order fixed in advance by the program; further, in the same time instant, it is impossible to 
execute more than one operation. 

To overcome the inherent limits due to the architecture of classical computer systems, it may be 
very convenient to use computers based on non - Von Neumann architecture, realized according the 
principles of parallel processing. 

Parallel processing is an efficient form of information processing which emphasizes the 
exploitation of "concurrent activities" in the computing processes. In other words, the concept of 
parallelism refers to the ability to overlap or perform simultaneously many concurrent activities. 

These concurrent activities are attainable in a computing system at various processing levels: job 
or program level, task or procedure level, interinstruction level, intrainstruction level. 

The most important ways of introducing parallelism into the architecture of computers may be 
summarized as: 

i) pipeline processing: the arithmetic/logic unit (ALU) and the control unit (CU) are organized 
as in a typical assembly-line; more precisely, their organization is such that each 
operation is subdivided in a number of steps, each step consisting in the execution of a 
suboperation. The steps relative to the execution of a given operation are suitably chained. In 
this way a number of operations can be, partially, overlapped thus exploiting a "temporal" 
parallelism; consequently the ALU and CU performance can be remarkably improved; 

ii) array processing : an array processor is a synchronous parallel computer with multiple 
arithmetic-logic units, called processing elements, under common control, that can operate in 
parallel in a lockstep fashion; 

iii) multiprocessing : several independent programs or tasks can be performed by a 
multiprocessor architecture containing two or more processor units, which share access to 
common sets of memory modules, I/O channels and peripheral devices. 

Pipeline and array processors are very suitable for vector computation, where component 
operations are data independent and may be repeated many times. In fact an array computer is able to 
perform an operation simultaneously on a large number of different data, while, due to overlapped 
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instruction and arithmetic execution, it is obvious that a pipeline machine is better tuned to perform 
the same operations repeatedly through the pipeline. 

A vector-pipelined computer has a parallel structure which, roughly speaking, deals with a vector 
operand essentially in the same way in which a serial machine deals with a scalar operand. This is 
possible since the general architectural model of a vector computer is based on several and 
independent functional units - of pipeline type -, each of which is dedicated to execute a specific 
operation (addition, multiplication, division, square root, logical operations, etc.) and can operate on 
different data in a parallel way. Moreover the central memory is organized in several independent 
memory banks, by which it is possible to execute contemporary access. 

The software production for this type of machine is characterized by the use of vector high level 
language, in particular extended FORTRAN language, characterized by "vector" facilities, which allow 
very efficient operations on vectors and matrices. 

However vector processing represents a low level of parallelism, because the concurrent 
activities are exploited at level of vector operations, whereas, as far as the machine architecture is 
concerned, there is still a strictly connection with the structure of Von Neumann computer. 

An high level parallelism is attainable by multiprocessors and distributed systems, whereby 
concurrent activities can be exploited and executed at task or procedure level. 

Concerning some of the most important aspects of future parallel processing, it is worth while to 
mention the effort currently done to develop parallel programming styles based on "non-procedural" 
models of computation, such as functional programming, data flow computation, logic programming 
and object-oriented programming. In fact, the present parallel computers, even if show new 
architectural aspects, are equipped by software tools still of "procedural" type and realized on 
purpose for Von Neumann computational model. 

3. PARALLEL NONLINEAR OPTIMIZATION 

The availability of parallel computers represents a good opportunity in order to improve the 
solution methods of non linear optimization problems. 

At present a great number of optimization algorithms exists, based on excellent theoretical 
framework and for which satisfactory computational results have been obtained when using sequential 
machines on reasonably complex problems. 

However it is very important to point out that the numerical techniques for solving non standard 
problems are strongly insufficient. In fact, mainly in the field of applications, there are problems, 
whose solutions are sought via conventional computers, which can cause severe computational 
difficulties; some of these are related to the following aspects: 

a) computing speed not sufficiently high when a real time or a global solution is needed; 

b) excessive memory locations keeping, and rounding errors propagation, typically due to the 
large scale dimension of the problem (thousands of variables and several thousands of 
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constraints) which may also cause an intolerable time consumption. 

In this context, parallel processing methodologies can bring considerable advantage, allowing the 
implementation of more complex and sophisticated algorithms which may exploit more powerful and 
original theoretical conceptions. 

Unfortunately in the past two decades there has been a limited amount of experience with nonlinear 
optimization algorithms on parallel computers. 

Only in the field of unconstrained optimization some authors proposed parallel variants of 
classical sequential algorithms, in a period long before parallel computers became commercially 
available. 

Among the initial attempts, Avriel and Wilde [3] as well as Karp and Miranker [4] developed a 
parallel Fibonacci algorithm for the linear search; Chazan and Miranker [5] proposed a conjugate 
direction method is which geometrically parallel directions are explored by parallel linear 
searches. 

Pierre [6] studied the Newton-Raphson method using the parallel computing facilities to generate 
finite-difference approximations to the first and second derivatives. Straeter [7] was the first author 
who proposed parallel variable-metric algorithm; his idea has recently been generalized by Van 
Laarhoven [8] . Finally, it is worth mentioning that Straeter and Markos [9] as well as Kowalik 
and Ramakrishnan [10] described a parallel version of the Jacobson-Oksman [11] method based on 
the so-called homogeneous model function. 

In very recent years a considerable amount of numerical experiments on parallel computers (in 
particular ICL-DAP with 4096 processing, elements), have been carried out by Dixon, Patel et al. 
[12], [13], [14], [15]. They developed, essentially, the idea of replacing the traditional 
one-dimensional searches by multi-dimensional searches, exploiting the parallelism at level of 
simultaneous evaluation of the objective function and its gradient at various points. 

Concerning the application of parallelism in order to solve nonlinear constrained optimization 
problems, Dayde [16] has recently carried out a work in which he solves a particular class of 
primal and dual structural-design problems by using the parallel uncontrained optimization methods 
of Straeter [7] and Jacobson-Oksman [11]. The solution of large scale optimization problems via 
decomposition and parallel computing is currently under investigation by De Silva et al. [17]. 

Finally, Schnabel [18] and Lootsma and Ragsdell [19] proposed two interesting general surveys 
about parallelism in optimization. 

In spite of their interesting attempts, the above mentioned research works have not yielded 
substantial contributions in order to exploit the real potentialities of the parallel processing applied 
to the solution of nonlinear optimization problems. 

This partially depends on limited production of computational testing, which is, principally, 
ascribed to the short availability of parallel machines. 

Another serious complication is due to the fact that there are several architectural models 
of parallel computer, each with its own particular advantages and disadvantages. Thus, to detect the 
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best parallel algorithm for a given problem, according to a performance criterion, becomes very 
difficult, because it depends on the particular architectural model used. 

Sometimes, the parallelism has, essentially, been exploited at level of simultaneous evaluation, at 
several points, of the objective function, its gradient and, in some cases, constraint functions; this 
represents a low level of parallelism without changing, in general, the mathematical structure of 
algorithms. 

A more deep mathematical analysis of sequential algorithms, together with a proper computational 
testing, can give a remarkable contribution to the design of parallel algorithms. However, to obtain a 
substancial progress in the parallel nonlinear optimization it is still necessary to develop the 
research activity in several directions. One of these directions concernes special cases and special 
structures, for instance large scale problems with interesting structural properties, which can be, 
efficiently, exploited by using parallel processing. But, only when a suitable "programming 
environment" for parallel processing will be available, it will be likely to realize a drastic "jump of 
quality" in the parallel optimization computations. 

4. 1 HE. OPTIMAL POWER FLOW PROBLEM 

An electric power system, aimed to fully satisfy, at any time, the electric energy demand, must 
achieve the fundamental goal of generating, transmitting and distributing electric energy, pursuing as 
priority objectives the economy, the reliability and the security of the electric service. 

To meet these extremely important requirements, the system planning and management activities 
must be carried out in a way not only consistent and appropriate but also suitable to achieve the best 
performance. The formulation of a mathematical model of the system is then the basis for a sound and 
effective decision support; it, also, allows to naturally devise an optimization problem in which 
decision variables are, generally, bounded by limits of technical and management type, of security and 
energy balance. 

These problems, known as Optimal Power Flow (OPF) problems, are formulated as nonlinear 
programs, involving both equality and inequality constraints, of the following general form: 



min 


/(x), x £ lR n 






h(x) = 0, 


h : H? n — ► 


lR m , 


m < n 


9(x) < 0, 


g :TR n — ► 


1RP, 





where: 

x € TR n : vector of electric quantities, whiqh are the decision variables; 

/(x) : objective function; it assumes several meaning according to the goals to 

pursue; 
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h(x) = 0 network equations (load flow equations); they represent the energy balance in 
electric network for a given configuration of the network; 
g(x) < 0 : technical, management and security constraints on the main electric quantities 

involved in the network. 

The explicit formulation of an optimal power flow problem, arising from a power system of 
great practical importance, is given in the sequel. For this purpose the electric power system is 
modelled as a network made up by nodes, which represent energy sources or loads, interconnected by 
branches which are models of the transmission lines. 

The number of network's nodes and the number of lines are denoted by: 

N = total number of nodes, 

N g = number of nodes where the active power is generated, 

N' = number of nodes where the reactive power is generated, 

N l = number of lines. 

The nodes are characterized by the following parameters: 

P g = N g - dimensional vector representing the active power generated, 

Q g = N' g - dimensional vector representing the reactive power generated, 

P d = N - dimensional vector representing the active power demanded, 

Q d = N - dimensional vector representing the reactive power demanded, 

V = N - dimensional vector representing the magnitude of the voltage at the 

generic node. 

S = (N-1) - dimensional vector representing the phase angle of the voltage at the generic 
node. It is assumed that the voltage phase of the N-th node is given, for example 

s N = o. 

Y = (N x N) - dimensional symmetric matrix representing the magnitude of the nodal 

admittances. 

0 = (N x N) - dimensional symmetric matrix representing the argument of the nodal 
admittances. 

The economic aspect of the network operation is of fundamental interest in our context. From this 
point of view it comes out, essentially on experimental basis, that a quadratic model for the cost of 
the entire active power generation is appropriate. To be precise, this cost, namely C, can be 
satisfactorily represented in the following form: 

C = P T g D(a i )P g + b T P g + Ie T i c. i = 1 N g , 



(D 
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where: 

D( aj ) = diag ( aj ), e { = i-th vector of the canonical basis in TR Ng . 

Moreover the parameters a, b, c, which are vectors of dimension N g , depend on the specific 

network considered and can be obtained by appropriate measurement. 

However, it has to be taken into account that the power generated must be such as to satisfy the 
following constraints: 

G p g - p d “ P < V ’ * ) = ° (2) 

G'Q g • Q d - Q(V, & ) = 0. (3) 



In these equations, which represent the energy balance in the network, G is a N x N g matrix, 
whose element gy is equal to one if the i-th node in the network is the j-th generation node, 
otherwise is equal to zero; analogously, the ( N x N' g ) matrix G' plays the same role for the 
reactive generation. 

In (2) and (3) the terms P(V, S ), Q(V, S ) represent the power lost in the network; their 
generic components have the following expression: 



P i = v i v k Y ik *i ' &k ' e ik)- ' = 1 . • ■ 


. , N 


(4) 


Q i “ V i s k v k Y ik cos ( ' & k ‘ e ik)- ' * 1 - • • 


• , N 


(5) 



where the sum is extended to all nodes k directly connected to the generic node i. 

The following constraints must also be satisfied: 



pm 

g 


* p g 5 pM g 


(6) 


Q m g 


< Qg < Q M g 


(7) 


V m 


< V < V M 


(8) 


s m 


< S < S M 


0) 



The previous inequalities are nothing but technical constraints on the main electric quantities 
involved in the network and depend on its intended purpose. 

For a network of general type, it is also necessary to consider the following "security" constraints: 

S(P g ,Q g )<S M (10) 

where S(Pg, Qg) is the Ng-vector whose generic component has the following expression: 



p g{ V q 9 ' ‘V S i' i = 1 N < 



(ID 



and 
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I (V, S ) :£ I M . (12) 

It is worth noting that the first member of (9) represents the square of the apparent power 
globally generated in the network; it is natural that the apparent power be upper bounded by S M . 

Moreover, the current in every branch must be also limited, as indicated by the constraint (12). 

The quantity I (V, S) is the Nj_-vector whose generic component has the following expression: 

I|(V, S) = [V 2 j + v 2 k - 2VjV k cos( Sj - s k )] / Z 2 i ; (13) 

the meaning of the scalar Ij(V, $ ) is the squared modulus of the current in the i-th branch whose 
impedance is Zj. 

Finally it is useful to introduce a limitation on the range of values covered by the voltage phase 
displacement between two adjacent nodes. If S (i) and S(j) denote the phase of voltage at the 
adjacent nodes i and j, then this limitation can be written as follows: 

| S(i) - S(j) (,<; A M , (14) 

where A M is a N L vector representing the upper bound on the phase displacement. 

Since, for the sake of economic reasons, the important target is to minimize the power 
generation cost, accordingly to the network constraints, it is possible to formulate the following 
nonlinear optimization problem: 

min { P T g D(a|) P g + b T P g + I; e T , c } (15) 

s.t. 

GPg - P d - P(V, S ) = 0 

G'Qg - Q d - Q(V, S ) = 0 

V m < V < V M 
pm g < Pg < P M g 

Q m g ~ QM g 
& m < S < 

S(Pg, Qg) < S M 

i (v, s ) < s M 
| S(i) - S(j) | < A M . 

This nonlinear optimization model is characterized by several aspects, which represent a serious 
complication for its numerical solution: 
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- the constraints are highly nonlinear; 

- the number of variables and the number of constraints, for a typical, significant, realistic 
application are very large ( thousands and several thousands); 

the most number of constraints are inequality constraints. 

Therefore, the particular structure of the OPF problem (specially the fact that the problem is 
highly nonlinear and constraints are mostly inequality constraints) suggests to use a method of 
solution possessing excellent theoretical properties (e. g. local and global convergence) particularly 
suitable in those difficult situations depending on the loads of the network. 

5. THE VARIABLE METRIC METHOD 

An early approach to the solution of problem (15) was based on a gradient-type method, the 
Sequential Gradient Restoration Algorithm, proposed by A. Miele et al. [1]. Although quite effective 
for small size problems and useful (all iterates belonging to the feasible region), nevertheless it 
proves too slow for successfully solve large scale problems in essentially real time. In addition, its 
convergence is not guaranteed in general situations. 

As previously pointed out, many factors make the solution of OPF problems very difficult 
computationally. For this reason it has been considered advisable to use the Variable Metric Method 
(VMM) proposed by S. P. Han [20] and M. J. D. Powell [21], which has proved very sound 
theoretically and potentially capable of remarkable computational performances. A review of it is 
shortly provided here. 

The constrained optimization problem: 

min F(x), F : TR n ► 1R (16) 

s.t. 

Cj(x) = 0 i € E 

Cj(x) >0 i € I, Cj : TR n ► TR , Vi 

is solved as a sequence of quadratic programming subproblems, based on function and gradient 
information at the current estimate x^ of the solution x* . The generic subprogram is: 

min Q(d) sf(x k ) + d T 7F(x k ) + y d T B k d (17) 

S.t. 

Cj( x k) + d T 7 C| (x k ) = 0 , i £ E 

Cj(x k ) + d T 7 Cj (x k ) 5 0, i £ I . 

Its solution defines, at any iteration k, the search direction d k and then 

x k+1 “ x k + “k d k • ( 18 > 

The steplength parameter a k G 1R + , extremely important for forcing convergence of the sequence 
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{x k } to x* , is provided by a suitable line search which: 

(a) must reduce the objective function, 

(b) must satisfy the constraints. 

The matrix B k in (17), intended to approximate, in quasi-Newton sense, the second derivative 

matrix of the Lagrangian function, involves the Lagrange multipliers of the quadratic subprogram, 
which approximate those of the original problem. 

The choice of a k is crucial. Among the different strategies on which the line search is based, an 

augmented Lagrangian type line search function has been considered here, according to 
Schittkowski [22] . 

It is worth noting that the method is mathematically very sound, since global and Q-superlinear 
convergence properties are proved [23] . 

6. VECTOR SUPERCOMPUTING 

An inherent limitation in computational performance affects available optimization methods 
(including the variable metric method, in spite of its sound properties) when they are used on 
conventional serial computing systems. In fact when solving a nonlinear program for a network model 
represented by (15), the large (and also very large) dimension is a typical feature and also the real 
time solution is required. A "time horizon" of minutes, typical for getting the benefits of optimizing 
the economic dispatching of electric energy, makes this type of solution very crucial. 

In these situations, very difficult to overcome with existing methods and standard sequential 
computational techniques, the parallel computing approach could be very useful to achieve a 
substantial improvement in performance. 

The possibility to detect parallelism in the OPF problems depends, above all, on the topology of 
the electrical network, the structure of the optimization model and the algorithm used for solving the 
required problem. 

As regards the topology of the network, some decomposition techniques (e.g. tearing operations on 
the network) may be applied to exploit parallelism. For each obtained subnetwork, an independent 
OPF subproblem can be solved, simultaneously with each other, to obtain a "partial" solution . On the 
basis of these solutions, a "whole" solution can be found, successively, taking into account the 
electrical conditions of tearing. 

This type of parallelism can be, efficiently, implemented in a multiprocessors or multicomputers 
environment, with "master-slave" processors organization. The "slave" processors carry out, in 
parallel, the independent OPF subproblems, whereas the "master", once received the "partial" 
solutions from the "slaves", computes the "whole" solution of the entire network. 

Although appealing, this naturally "parallel" approach may suffer for the present lack of 
concurrent algorithms conceived for successful! implementation on computer systems with highly 
parallel architecture. 
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However a lower type of parallelism can readily be exploited by using modern, commercially 
available, vector supercomputers. 

In these machines the parallelism is low and is performed essentially at arithmetic level by 
exploiting the vector and matrix operations. In fact the peculiar characteristic of vector processing is 
the capacity to deal with a vector operand, i.e. an ordered set of N elements, where N is the length of 
the vector, in the same way in which sequential computation deals with a scalar operand. 

To examine the efficiency of vector processing over scalar processing, it is worthwhile to compare 
the following two programs, one written for vector processing and the other written for scalar 
processing. 

In a conventional scalar processor, the FORTRAN DO loop 
DO 1 I == 1 , N 
A ( I ) = C(I) + D(I) 

B(I) = S + A(I) 

1 CONTINUE 

is implemented by the following sequence of scalar operations: 

INITIALIZE I =1 
1 LOAD C(I) 

LOAD D(I) 

ADD C(I) + D(I) 

STORE A( I) <- C(I) + D(I) 

LOAD A(I) 

LOAD S 
ADD S + A(I) 

STORE B(I) <— S + A(I) 

INCREMENT I <— I + 1 
IF I < N GOTO 1 
STOP 

On a vector computer the above DO loop operations can be vectorized into two vector instructions 
in a sequence: 

A(1 ; N) = C(1 ; N) + D(1; N) 

B(1; N) = S + A(1; N) 

where the notation A(1;N) refers to the N-element vector A(1), A(2), . . ., A(N). 

It can be easily seen that the execution of the scalar loop repeats the loop-control overhead in each 
iteration. In vector processing using pipelines, the overhead is reduced by using hardware or 
firmware controls. 

One of the fundamental motivations for implementing the VMM on a vector computer is based on 
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the consideration that the amount of computational work is strongly affected by the dimensions of 
vectors and matrices playing a decisive role in the execution process of the algorithm. 

Moreover, the vector machine makes possible to code the algorithm without any drastic 
reformulation or change of its conceptual structure, overcoming the difficulties wich might arise in 
implementing, for instance, decomposition techniques in a multiprocessing environment. 

A typical feature of vector processing consists in the matrix product execution. From a 
mathematical point of view, the product of two matrices A and B (N x M and M x N dimensions 
respectively) is a N x N matrix C, whose elements are related to A and B elements by the 
following expression: 

M 

C(I,J> =Z { A(I,K)xB(K,J)}, I , J = 1 , N 
k=1 

In a conventional serial computer the above equation can be implemented using FORTRAN 
language, as follows: 

DO 3 I = 1, N 
DO 2 J = 1, N 
DO 1 K = 1,M 

C(I, J) = C(I, J) + A(I, K) x B(K, J) 

1 CONTINUE 

2 CONTINUE 

3 CONTINUE 

where it is assumed that all elements C(I, J) are set to zero before entering the code. 

In this way each element of C is computed sequentially as scalar product between the rows of A 
and the columns of B. 

In a vector computing environment it is possible to rearrange the code in such a way, for instance, 
to compute each column of C as a linear combination of columns of A having the elements of the 
respective column of B as coefficients. The code is, therefore, as follows: 

DO 2 J = 1, N 
DO 1 K = 1, M 

C( , J) = C( , J) + A( , K) x B(K, J) 

1 CONTINUE 

2 CONTINUE 

where C( , J) and A( , K) indicate the columns of C and A respectively. In the above code there is 
a parallel addition of N elements and the multiplication of scalar B(K, J) by the vector A( , K). 
Therefore the parallelism exploited is N (the elements of each column of C), compared with 1 for 
the original conventional code. Furthermore, this type of operations (vector + vector * scalar) are 
very suitable to be executed on the vector computer (’’chaining" of the operations) and it is possible 
to reach high value of performance. 
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For computing the solution of OPF problems whose model is the nonlinear program given by 
(15), a vector supercomputer CRAY X-MP/48 has been used. 

The CRAY X-MP/48 is a multiprocessor machine (four processors), in which each processor 
has a pipeline architecture (same organization as CRAY 1); the four central units share a large 
core memory unit with interleaved organization (32 banks) and I/O subsystem; the clock cycle is 
about 8.5 nsec. 

In addition to vector capabilities, the multiprocessor organization of CRAY X-MP allows to use 
efficient multitasking facilities at different programming levels (parallelism among independent 
subroutines or do loops). 

7. NUMERICAL RESULTS 

The OPF problem (15) has been taken into consideration for numerical computation in the case 
of the following networks: 



NET 1 


N = 5 


CD 

II 

_l 

z 


N g = 2 


N' g = 2 


NET 2 


N - 10 


N l = 13 


Ng = 7 


N’g = 7 


NET 3 


N - 57 


N l = 78 


N g = 4 


N'g=7 



The VMM, in its sequential implementation, has been firstly used on the serial computer IBM 
4341; the relevant numerical results are reported in Table 1. On the other hand, in Table 2 the 
results obtained by using a parallel implementation on the vector supercomputer CRAY X-MP/48 
are displayed. 



TABLE 1 



NETWORK 


N 


N v 


N C 


h 


F* 


CPU 

sec 


NET 1 


5 


13 


56 


5 


.1 38x1 0 3 


5 


NET 2 


10 


33 


132 


19 


.280x10 


91 


NET 3 


57 


124 


600 


5 

low acc. 


.339x10 


38 x 60 




83 



TA BLE; Z 



NETWORK 


N 


N v 






F* 


CPU 

sec 


NET 1 


5 


13 


56 


5 


.1 38x1 0 3 


0.122 


NET 2 


10 


33 


132 


19 


.280x10 


1.60 


NET 3 


57 


124 


600 


8 

low acc. 


.379x10 


34.8 



In the above tables the following notation is used: 

N = number of nodes 

N v = number of variables; 

N c = number of constraints; 

I T = number of iterations; 

F* = function value at the computed minimizer. 

For the sake of comparison it is worth noting that a remarkable speed-up factor was gained by the 
vectorized algorithm versus the standard implementation, in terms of CPU time. 

The improvement attainable with the vectorized algorithm may be particularly valuable when the 
network's dimension increases. Thus, a special attention must be devoted to the way in which the code 
is designed and matched to the architecture of the machine; in fact, only under these conditions, its 
capabilities can be conveniently and effectively exploited by the complex data-structure typical of 
large scale problems. 

It can be drawn the final consideration that the optimal operation of a complex electric network, 
even rather large, is successfully computed by a vector machine, thus making possible its "real 
time" management . The use of a sound nonlinear optimization algorithm is a crucial contribution to 
pursue this objective in a reliable way. 

8. CONCLUDING REMARKS 

Looking at the numerical results previously reported, the conclusion can be drawn that the 
performance of the parallelized VVM algorithm is sharply superior to the sequential one. In 
particular, the speed-up factor indicates a "jump of quality" in favour of the vectorized 




84 



implementation. 

This aspect encourages to face bigger problems, whose dimensionality is usually prohibitive for 
sequential machine implementations. 

However, the occasion of this initial attempt to solve OPF problems by using new computational 
techniques, allows some considerations useful for future works. 

The network model used, although quite general does not take into account further real aspects 
which would make more complex the model. 

At present the VVM may suffer for inconsistency of constraints in the quadratic programming 
subproblem, at least in some critical situations. 

Furthemore, it is not taken into account the sparsity structure of the matrices involved in the 
problem. 

Finally, parallel techniques have been applied at level of code design, without changing the 
structure of the algorithm. 

In perspective the potentiality of parallel computation may be fully exploited by designing 
optimization methods inherently conceived for implementation on multiprocessing computer systems. 

This may result in an extraordinary improvement of performances opening the possibility to solve 
even extremely large scale problems in almost real time. 
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Abstract 

This paper introduces the concepts of automatic differentiation and parallel 
processing and discusses how the advent of the ADA language on parallel processing 
machines may transform the solution of practical optimisation problems. 

1. Introduction 

The solution of large industrial optimisation problems should produce 
considerable economic and technological benefits. The mathematical theory 
necessary to write iterative algorithms for the solution of both unconstrained and 
constrained optimisation algorithms is well known and codes exist that regularly 
solve small to medium industrial optimisation problems. These codes are normally 
successful when applied to problems containing between 5 and 100 variables, but are 
normally not written with the intent to solve problems containing thousands of 
variables. Though occasionally they have been successfully used to solve such 
problems. There is really a surprisingly small amount of optimisation software 
available which efficiently handles the sparse matrices that nearly always occur in 
large problems. There is also apparently little industrial demand for such 
problems. 

If we seek for the reason for this lack of demand, I believe it exists not 
because of a lack of problems but because of a lack of confidence in the ability to 
solve such an optimisation problem. This lack of confidence partly arises from the 
difficulties people have experienced in solving more modest problems in the past. 
Assuming codes are implemented properly, these difficulties have tended to arise in 
three areas. Most optimisation algorithms that can theoretically be shown to 
converge need to be given not only accurate algorithms to evaluate the objective 
function and the constraints but also accurate algorithms and hence expressions to 
compute their derivatives. This can be difficult for simple test functions in a 
few variables but can take man months of effort on industrial problems. 

Once these derivatives are available two further difficulties have caused 
problems in the past, namely, the need for sufficient store on the computer to store 
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second derivative information; and the tendency for solution codes for industrial 
optimisation to require considerable computer time. 

This paper is written with the assumptions that computer store is no longer such 
a vital problem and that with the decrease in relative costs of computer time to 
human time, we can afford to increase the solution time on the computer to do away 
with the man months of effort often needed to obtain accurate derivatives. This is 

in any case reduced as the availability of accurate derivative information 
frequently reduces the number of iterations required by optimisation algorithms. 

The first topic to be treated in this paper will therefore be automatic 
differentiation. This technique introduced by Rail [1] enables accurate derivates 
to be obtained by the computer. 

Whilst simple implementations can considerably increase the computer time, the 
use of the principle of partially seperable subfunctions, Toint [2], can reduce this 
considerably. In this aproach we decompose 

F(x) = £ f.^) 
i 

where each subfunction f ± only depends on a small number of variables x i . 

One of the most natural decompositions occurs in finite element optimisation 
Dixon & Ducksbury [3] 

F(x) = Z F el (x el ) 
el 

where the structure for automatic differentiation is identical to that used 
successfully on the ICL-DAP parallel array processor, Ducksbury [4]. 

2. Automatic differentiation 

2.1 Disadvantages of analytic or numeric approximation method 
Most algorithms for solving unconstrained problems 
Min F(x) x c R n PI 

or constrained problems 
Min F(x) x e R n 

s.t. e.(x) =0 i = 1 ... M e P2 

h.(x) > 0 J = 1 ... M x 

theoretically require the algorithm to methods for evaluating first derivatives 
accurately i.e. 




and and when appropriate. 

3x k 

As the same arguments apply to both PI and P2 we will only discuss the simpler, 
unconstrained problem PI in this paper. 
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For this problem it is veil known that the availability of second derivatives 

r 3 2 F 

G U = 

3x a 3x j 

can produce more efficient iterative methods. 

In even 20 dimensions to obtain g accurately implies analytically obtaining 20 
difficult formulae and then programming each accurately. To obtain the second 

derivatives 190 expressions would need to be programmed. Each program would need 

to be verified. 

It has therefore become customary to obtain numeric approximations to the 
derivatives, by using, say, central difference formula 
g k = [F(x + ha.) - F(x - ha. )]/2h. 

These formula are of course subject to both truncation errors (0(h 2 )) and rounding 
errors (0(l/h)). It can therefore be difficult to choose an appropriate value of 

h. If we are asking g£ to verify an analytic expression g k and the two disagree, 
it is frequently difficult to decide whether this is due to an error in calculating 
g k or the selection of an inappropriate value of h. 

The use of automatic differentiation involves less arithmetic than the 
calculation of g k and avoids these difficulties. 

2.2 Rail's method of automatic differentiation 

The method of automatic differentiation introduced by Rail calculates the function 
value, gradient vector and Hessian matrix simultaneously. 

Each entity y occurring in a program is stored as a triplet:- 

Yi = (y*. zj, yp 

where y' is an n vector k = 1 . . . n 

3x k 

and y" is an n x n vector ^ ^ k = 1 ... n 

3x k 3x x 1 = 1 ... n 

i. e. a list of n 2 + n + 1 elements. 

An algebra of triplets can now be defined which mimic the rules of 
differentiation 

1) u + v = (u, u' , u") + (v, v', v”) 

= (u + v, u' + v' , u" + v M ) 

2) u - v = (u - v, u' - v' , u" - v" ) 

3) u.v = (u.v, uv' + vu',uv M + uv' T + vu ,T = vu") 

4) u/v = (u/v, (vu' - uv' )/v 2 , (v 2 u" - v(v'u' T + u'v' T ) 

+ 2uv'v' t - uvv”)/v 3 

v + 0. 

Similar rules can be defined by all the standard operations 
exp(x), sin(x) , cos(x), log # (x) etc. 
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These rules imply that the natural variables are held as x. = (x., a., 0) and 
constants as c = (c, 0, 0). 

In Fortran the implementation of a function evaluation in triplets is cumbersome 
as each addition, subtraction, multiplication and division must be programmed by a 
separate subroutine call. At this point the use of concepts of types and extended 
operations in the ADA language greatly simplifies the programming. These can be 
not only used for well known constructs such as matrices, where the definition of a 
type matrix and an extended operator * can allow the product 

C = A*B 



to be programmed in one line, but also for new constructs such as Rail Triplets. 
Mohseninia [5] defines the type triplex and has written a full set of operator 
extensions to be applied to them. 

If XI, X2 and F are triplets then in ADA the statement 
F = 100.0*( (x - Xl**2)**2) + (1.0 - Xl)**2 



given XI = (-1.2; (1, 0) T 



X2 



(1.0; (0, 1) T 



0 0 
l 0 0 
0 0 
l 0 0 



calculates 

F = (24.2; (-215.6, -88.0) T ; 



' 1330 480 p 
s 480 200 , 



i.e. the function value, gradient and Hessian of Rosenbrock's function at the 
classic starting point. 



2.3 Automatic differentiation and optimisation 

Although we have described automatic differentiation in the previous section as a 
triplet, containing function, gradient and Hessian; it is obviously trivial to 
reduce it to a doublet consisting of function and gradient alone. 

It would obviously be possible to run any gradient method such as the variable 
metric method or conjugate gradient method for unconstrained optimisation or the 
recursive quadratic programming method for constrained optimisation by calculating 
the first derivatives using the doublet form of automatic differentiation. 

Similarly by using the triplet form second order methods such as the modified 
Newton method, or Dembo-Steihaug truncated Newton method [6,7] for unconstrained 
optimisation or Maany's QFQC [8] method for constrained optimisation can be used 
without difficulty. 

In practice, due to our experience reported in Dixon & Price [9] that the 
truncated Newton method is efficient over a wide range of variables, we have 
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incorporated automatic dif ferentiation into that code, Dixon & Price [10]. An ADA 
implementation is described in [5]. 

In their implementation of the truncated Newton method Dembo and Steihaug avoid 
the calculation of the matrix G by noting that this only appears in their algorithm 
in the form Gp and this can be approximated by 

Gp = (g(x + hp) - g(x) )/h. 

In Dixon & Price [10] a modification of the automatic differentiation triplet is 
described that uses the direction p as an input and evaluates a triplet {F, g, Gp) 
without using a matrix store. The algebra of such triplets is also given in that 
paper. 

It is however well known that sparse Hessians can frequently be computed by 
difference techniques in far less than n steps and it can therefore be argued that 
for such problems Dembo and Steihaug' s strategy of calculating Gp at every minor 
iteration would be far more expensive than calculating G initially in a few gradient 
calculations and then only forming the product at each iteration. The latter seems 
especially attractive if G is sparse. 

It therefore seemed sensible to seek an alternative triplet that would preserve 
sparsity. 

2.4 Sparse automatic differentiation 

Any objective function calculation must be made up of a number of subfunction 
calculations. These subfunctions will normaly each depend simply on a subset of 
the variables. The Hessians of these subfunctions will therefore usually be 
sparse. This is especially true if overall the Hessian is sparse. 

This idea has already been utilised by Toint [2] in his partially seperable 
function theory 

F(x) = Z f i (x i ) 
i 

and Dixon & Ducksbury [3] in finite element optimisation 

F(x) = E F.i ( x . ! ) . 
el 

Automatic differentiation has therefore been further modified, in that 
associated with each 'sparse triplet' definition is a list of the indexes of those 
variables that affect that subfunction. If there are K such variables, the 
gradient part of the 'sparse triplet' is a K vector and the Hessian part consists of 
K linked lists. These linked lists which use the Dynamic data structure and are 
updated using the recursive facilities available in ADA, are described in detail in 
[5]. In each row of the sparse matrix, the structure states in which column the 
particular nonzero element exists, its value and the position of the next element in 
the row. 
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By computing the Hessian in this sparse way the amount of arithmetic is kept to 
a minimum during the calculation and the products Gp used in the truncated Newton 
method can be computed very efficiently. 

Using the 'sparse triplet' approach the calculations involved in even small 
compact functions are reduced. 

We therefore now recommend the calculation of the objective function by the 
'sparse triplet' method at the start of each outer iteration of the truncated Newton 
method; and simply forming the product Gp in each inner iteration. 

We have not been able to show any relationship between the amount of arithmetic 
involved in such a sparse triplet calculation and the amount needed to form a sparse 
Hessian by the Curtis, Powell & Reid [11] method. This question is still open. 

Six combinations of automatic differentiation and the truncated Newton method 
have been investigated. Four in Fortran and two in ADA, namely: - 

1) Compute g and G by Rail's triplet method 

2) Compute g by Rail's doublet method and Gp by the finite difference approximation 

3) Compute g and Gp by the modified triplet, Dixon & Price [10] 

4) Compute g and G by 'partial separation' and submatrices, Dixon & Price [12] 

5) Compute g and G by 'sparse triplet' (ADA), Mohseninia [5] 

6) Compute g and Gp by 'sparse modified triplets', Mohseninia [5]. 

An automatic differentiation implementation of the Gauss Newton method for 
minimising a sum of least squares with small residuals is described in Dixon & Price 
U3]. 

Toint [14] gives an efficient code for large residual problems. This could be 
implemented using automatic differentiation very easily. 

3. Parallel Processing 

Many optimisation problems can be decomposed into subfunctions 

F = I 
el 

where 

VF = E VF el where VF #1 is essentially an n #1 x 1 vector 
el 

and 

^F = E W 2 F el where W 2 F #1 is essentially an n el x n el matrix, 
el 

This is of course a similar structure to that used in 'sparse triplet' automatic 
differentiation. This is the same structure used by Ducksbury [4] in finite 
element optimisation on the ICL-DAP. He was solving a set of nonlinear partial 
differential equations and converted this into a nonlinear optimisation problem. 

Both were then approximated using standard finite element techniques and each 
element was calculated on its own processor. 
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One sample problem considered was 



e(u) = 



3 2 u 



3x" 



3 2 u 



ax? 



„ 3u n 
Ru = 0 

3x, 



2 

s.t. 4<x 1 <5; 4<x 2 <5. 

The solution of this problem is u = 6x 1 /Rx 2 . 

The problem can be converted to an optimisation problem 
Min F(u) = JJ e(u) 2 dA. 



Obviously the global minimum of F(u) = 0 and this occurs at the solution to the 

partial differential equation. Ducksbury introduced 3 fields A, B, C 

a D 3u „ 3u 

A = u, B = , C = 

3x x 3 x 2 

and a function 



F = 
where 




e 



i 



3B a 

+ 




e 2 dA 



RAB = 0 



e 2 = B 



e 3 = C 



3A 



3x, 



dA 



= 0. 



Each field A, B, C was approximated by a standard bilinear element over rectangles, 
which led to N £L = 12 even for fine meshes with over 4800 variables. 

It is very intriguing to note that if standard linear elements had been used 
over triangular elements he would have had N £L = 9; and for triangular elements 
with quadratic approximations over u then N EL = 6. 

Ducksbury' s results on the DAP were obtained using analytic differentiation but 
the structure used in Automatic Differentiation, method 4, and parallel computation 
are identical. 

We can therefore predict a future where automatic differentiation in ADA will be 
implementable on parallel processing machines using its parallel tasking facilities. 



Conclusions 

Automatic Differentiation can be used to obtain derivative calculations for 
first order methods like variable metric and conjugate gradient methods. 

It can be combined very efficiently with the Truncated Newton method. 
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Efficient methods for computing sparse Hessian matrices of large problems can be 
easily implemented especially in ADA using the 'sparse triplet' approach. 

The method used for sparse matrix calculations of finite element optimisation 
problems are very similar to those previously used to solve such programs on the 
parallel processing array (ICL-DAP) machine. 

We can anticipate being able to combine automatic differentiation and parallel 
processing in ADA to solve large problems. 
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Preface - From the mathematical point of view mult icriter ia optimi- 
zation (MCO) is a natural generalization of optimization problems. The 
need of decision making in contradictory situations makes MCO methods so 
interesting for us. MCO deals with one of the most sophisticated aspects 
of human activity which is to achieve several goals by the single act of 
decision making. MCO models and ordinary optimization are not very much 
different in task definition, giving us hope to use the similar 
n umer i cal met hods . 

This paper gives the overview of the MCO package as one of the main 
parts of the DISO - dialogue system for optimization problem solving 
which was developped in the Computing Center of the USSR Academy of 
Sciences. Two MCO methods are described in this paper. Both methods are 
based on the idea of nonuniform covering technique and inclusion 
function approach, which was initially developped for global extremum 
search Cl-51. The complexity of MCO tasks makes it necessary to create 
effective numerical methods to find both a single point of Pareto set 
and an approximation of this set also. The paper describes two MCO 
algorithms, which differ in the interpretat ion of the solution and as a 
consequence in the complexity of numerical calculations. The /min 
features of the MCO package are described also. 

1. Overview! of the DISO system. 

The basic feature of the DISO system is the integration principle. 
Unlike other dialogue systems, of this class the DISO system includes 
several interconnected packages for the solution of the following tasks: 

- unconstrained minimization; 

- nonl inear programming; 

- opt imal control ; 

- 1 in ear programming; 

- global optimization; 
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- mult icr iter ia optimization; 

- linear algebra; 

- nonlinear algebraic equations. 

For all these tasks the DISO system delivers for the user the 
unified set of dialogue capabilities, which includes: 

- task definition and analysis in text mode; 

- automatic optimization class recognition and correspondent 
dial ogue pac k age initiation; 

- local analysis of task definition functions in a given point; 

~ changing of optimization method and its control parameters in the 
d i al ogue sess ion; 

- asynchronous control of the solution process which makes it 
possible to stop the process at any moment; 

- control of the hierarchical solution process with the automatic 
or manual cr eat ion of the subordinate tasks from the list 
mentioned above; 

- choice of the numerical or analytical differentiation schemes; 

- control of the numerical and graphical interaction in the process 
of solution; 

From the implementation point of viewi dialogue capabilities of the 
DISO system are based on the new approach of the multiwindow technique 
which includes the capability to access the values of the variables 
using special fields in the windows. The methods in each optimization 
class have different forms of graphical output in accordance with their 
basic mathemat ical schemes. 

The integrated mode of the DISO system makes it really powerf u.1 1 
instrument for the solution of different application tasks. It is 
essential, that the system makes it possible to change the task 
definition in the process of the solution and to transfer the task from 
one mathemat ical model to another. The typical example of such 
transf ormat ion takes place if you add a restrictions to the initial 
unconstrained optimization problem thus creating nonlinear programming 
problem. Another example: transforming all the criteria but one to the 
restrictions wiill change the MOO problem to the more simple class of 
nonlinear programming. It is important to note that all the numerical 
results achieved so far will remain accessible after these 
transf ormat ions . 

Most optimization models have hierarchical structure . The nonlinear 
programming methods, for example, may reduce the task to unconstrained 
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minimization on every iteration. Optimal control methods usually create 
subordinate tasks of nonlinear programming and so on. In all these cases 
the DISO system provides the dialogue capabilities of the correspondent 
class- for the solution of the subordinate problem. Finishing the 
solution will move the user again to the level of the initial problem. 
This feature of the DISO system proved to be really valuable for the 
solution of several difficult applied problems- saving time and 
increasing the accuracy of the solution. 

The MOO dialogue package plays a central role in the DISO system, 
because its mathematical model can be obviously treated as a 
generalization of the other optimization models listed above. 

2. Statement of MOO problem. 

The mu.l t icr iter ia problem that we consider has the following form: 

min Fix), (1) 

x <5 X 

where x£R is a vector of decision variables, X is the feasible decision 
set, F~CF^, - , . , F**^! is the objective vector, F: P^-^R^ vector~fu.net ion 
F is continuous on X. The decision set X is assumed to be a closed and 
bounded (therefore compact) . The goal is to find the efficient (Pareto) 
set X^ of X with respect to F, that is 

X # = <x£X: if F(w)4F(x) for some w£X then F(w)=F(x)>- (2) 

We shall use the following convention: if a,b£.R then a £ b if and only 

if a L ^b u for all i - 1 , s . 

We propose two extent ions of ^-optimality concept which were 
developed for scalar optimization problem to vector case. In the first 
extent ion we introduce ^-efficient set as follows: 

x‘;{x£x: F(x)«F(k j< )+£, i/>f-iere x^eX*}, (3) 

where vector £€.R has all positive components and is named accurasy 
vector . 

The set W^ is called £-net of the Pareto set iff: 

1) for any point x£.X^ there exists a point such that 

F(z)<:F<x)+£; 

e 

2) there are no two different points x and z in W such that 
F(x)$F(z) . 

Let X and P- be a compact right parallelepipeds parallel to the 
coordinate axis (abbreviated as a box in the sequel): 

X;{x£R*': a<x$b>, 

P.^xfiR 11 : a*^xj;b -> 3 P;CX, r\ b-fiR*l 

i u L *- t 
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The main diagonal of the box we denote as -a* , the midpoint of 

the box is c» =1/2 (b - +a; ) . 

Let's introduce the m-diment ional vector — function Q(P) , for which 
every j-th component is defined by the condition: 
c/<P> = min F^<x>, PCX. 
x 6 P 



We assume that for Q(P> it is possible to find vector~f unct ion G<P) 
which is the lower estimation of Q(P) on the box P. This function must 
satisfy two conditions: 

G(P)$Q(P>, (4) 

. ,1 im (G<P ; >-Q<R > >=®. (5) 

Here we introduced the sequence of the boxes which satisfies the 
following conditions: 

p i+i cp :’ ^4™ p i= p ^ fe x* 

where is accumulating point. 

For the given vector~fu.nct ion F<x) the vector-function G(P) can be 
found either on the basis of interval analysis C81 or by introducing 
some additional hypothesis. For example, supposing that all the 
components of F(x) on the X set satisfy the Lipschitz condition with 

constants = max max | F^ i<x)| than we have: 

X 



^ l^i^n x C. X 

G J (P. )=F*(c i )— l/2lf lld-ll^. <6> 

If in addition F(x) satisfies Lipschitz condition with constant M then: 
g'<P ; )=F / (c- >- l/2ild ; lL,min«:i/ , II F 1 <c;>ll. +1/4M <7> 

c L coo x t JL 1 «o 

It is obvious that these functions G(P) satisfy conditions (4), (5). 



3. Description of MOO methods. 



Now we are going to describe two algorythms for the approx imate 
solution of the problem (1). During the computation process the 
algorythms. will generate the sequence of the boxes 

B *“ <P JL 3 P X 3 ---* P k > * a11 P i C x " 

Let each box P^ be linked with the structure S^=(c^, d*, G -) , where 

G. =G(P» ) . We call the set S for the sequence the structure list 
t l K 

/-; /c r- C \ 

O— a m m m 3 w m 

Algorythms differ in the interpretat ion of the solution and time 
consuming. W* set is obtained by one of them (second algorythm) and a 
single point from the X^ set - by another (first algorythm) . The general 
scheme of both algorythms is described below. 
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Algorithm. 

Initial actions: 

1) . Let F^rX* calculate c^,* 



F(c^) * G^=G(F*^> and set 






3=-CS 1 >. 



Main cycle: 

2> . Choose the box P. from E , for which min max G; is achieved. 

> K J.4L4K 4.4+4.* C 

3). Choose a coordinate direction t in box P^* parallel to which P^ 



an edge of maximum length* i.e. d. 



max d L . Bisect R. in the 
14 U K * * 



direction t* getting boxes P^* P with midpoints c^* c^ and 
d i agonal s d w * d* respect i vel y . 

et f 

4) . Calculate F(c^)* F(c^) * modify W* or X^ and define the vector values 

G^-G ( P^ ) * G^ — G ( F^ ) . 

5) . Remove the box F^ from the sequense B^i.e. remove the structure 

from the list S'. Add into the list S two structures: 3 , ~ ( c, * d,* GJ> 



and S^r ( c^, * 



assuming b. 






6) . For all S* from the list S check the following condition: if F(x*>^ 

^ £ d 

G(P*) + £ for any x: from LL or XT: then remove S» from the list S and 



remove P« from B K . Order new 1 ist -C5 
n ame Sr < S ^ 3^ < • 



... * Sf. > and give it the 

L r 



7). Let k=p. If k<£0* i.e. S is not empty then goto 



Con c 1 ud i n g oper at ion s : 

8). Invoke output procedure. Stop computat ion . 



4. Problem solving in MOO package. 



The task below was solved in order to investigate the quality of 
MCO algorithms.. Full definition of this task is given in C7 3. The 
solution based on the sequence of minimization subtasks was found there. 
This task includes two criteria* five parameters and has the following 
form: 

min FCx)* 
x e X 

n x 1 + 1 n 

lAhere F A <x> =1- j~ j (l- (l-r l ) j . F <x) = Jh I x c , 

i-1 i”l 

X=<x t ^ N: 0$x l £10* i=l*n>* nr. 5* vectors c and r are given in the Table 1. 

Two solutions were obtained using both algorithms, with different 
accuracy vectors £. G vector function was defined in accordance to (6) . 
Lipshits constants vector was chosen to be Lr E 0 . 7 0.73. This value is 
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5 . MCO pac k age dial ogue capab i 1 it ies . 

As was- already mentioned above , the DISO system provides the user 
the unified set of dialogue capabilities to control every step of the 
solution process from the task definition up to the analysis of the 
numerical resul ts . Let •' s overv iewi br ief 1 y these capab i 1 it ies . 

The MCO task definition can be done using text processor which is 
part of the DISO system. A special language DIFALG is used for task 
definition. This language is very similar to the ALGOL- SO. The inner 
form of the function representat ion is created as a result of the task 
definition compilation process. The calculation of the numerical values 
of the functions in a given point is based then on the interpretat ion of 
this inner form. The peculiarity of the DIFALG language is that its 
semantics includes the notion of the differentiation. Function 
evaluation may be done in parallel with the first and second derivatives 
of this function in a given point by the user request. The important 
point is that it gives the user not the numerical approximation but the 
exact value of the derivatives, which corresponds to the analytically 
evaluated value. The differentiation algorithms, are based on a special 
highly effective approach which qualitatively increases the calculation 
speed . 

Task definition includes comment lines. The system automatically 
defines the optimization class and passes the control to the 
correspondent dialogue monitor analysing these comments. For example, to 
define MCO problem, the user marks with the special comment those 
functions in the DIFALG listing which he wants to be included to the set 
of the criteria. The other comments mark the functions to be included to 
the equality and nonequality restriction sets. If any, the problem will 
be classified as nonlinear (single or mul t icr iter ia) programming model. 
Special comments define the mode of optimization: local or global, give 
initial point value, paral 1 el ep ipedal restrictions in the parameter 
space etc. The user may return to the task definition step at any moment 
of the solution, make same modifications and continue the solution 
process . 

The user initiates the dialogue session finishing the task 
definition. Tine set of control windows- become available to him at this 
moment. Each window provides the user with some resources to control the 
solution process. The MCO dialogue monitor opens the access to the three 
windows . 
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The first one provides the capabilities for manual analysis of the 
functions local properties in a given point. It also controls the MCO 
method choice, initiation and termination. As was mentioned above, the 
DISO windows, are structured in a sence that they include the different 
sets of fields to access the variables and control their values. The 
first MOO window contains the fields for the decision vector, criteria 
values vector, list of methods available etc. The user may move the 
cursor to the parameter vector and give this vector some initial value. 
The cursor movement to the criteria field automatically yields in the 
recal cu.l at ion of the criteria functions in a given point with immediate 
output of the numerical results in this field. The field «Ahich is 
connected with the list of available methods plays the role of the menu. 
Each method has its own 1 ist of control parameters. The choice of the 
method automatically provides the user with the new list of fields for 
control parameters which appear in the window and become accessible by 
the ordinary routine of the cursor movement. Another field is 
responsible for the type of the solution: the user can choose 
t -approx imat ion of the Pareto set or single-point solution as it was 
mentioned above. Finally, the window has the menu field to run the 
method, stop it and switch to the task modification mode. 

The second window provides the user with the view* of the Pareto set 
in the process of its creation. With the help of the fields in this 
w»indow* the user can point out two coordinate axis to define the 
two-d iment ional plain to create the projection of the pareto set. The 
solution process will show* each new* Pareto point in the special 
graphical field of this wiindow*. Finishing the solution process the user 
may enter the mode of Pareto set analysis. The cursor wiill take the form 
of the arrow* in the Pareto field. The user can move this arrow from one 
Pareto point to another, visualising correspondent numerical values of 
criteria and parameter vectors. The user can store the Pareto set in a 
file with a given name or retrieve previously defined Pareto set and 
continue the solution to achieve more accurate Pareto approximation. 

The third window* gives the possibility to visualise the covering 
technique in the parameter space. Each method has its own covering 
strategy. The user can vizually estimate the efficiency of the chosen 
covering scheme for the given task to be solved. 

The system provides the possibility of asynchronous control which 
includes, for example, switching from one wiindow* to another while the 
optimization process continues it's progress. 
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Conclusion. The MCO methods described in this paper lAiere tested 
using several tasks and proved to be rather effective. There exist 
several reasons of the methods success. First, unlike the known methods, 
these methods are strongly oriented on the effective use of the computer 
memory which decreases the amount of function evaluation. This is most 
valuable feature if function evaluation takes long time. Second, the 
interval analysis technique makes it possible to eliminate the initial 
Lipshitz constant estimation. Third, there exists a simple and natural 
way to organize parallel calculations by the feasible domain division 
between several processors. Fourth, the described methods permit the 
inclusion of local search algorithms-, which may speed the calculations 
enormously. Fifth, the second of the proposed methods creates the Pareto 
set using nonun if ied one-way covering technique, instead of the common 
approach, which is based on the manifold solution of the auxiliary tasks 
on t he f eas i b .1 e dorm i n . 

Dialogue MCO package, which is part of the DISO system, is a 
promising numerical basis for the implementation of numerous decision 
support systems. 
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1. Introduction 

In [8, 9, 10, 12, ] successive overrelation methods are proposed for solving the dual of 
the problem of finding the least 2-norm solution of a linear program. This leads to an 
exterior penalty formulation of the dual of the original linear program with the interesting 
property that the penalty parameter need not approach zero in order to obtain an exact 
solution of the primal linear program [1, 14]. Thus the penalty parameter need only be 
less than a certain threshold value in order to obtain an exact solution to the primal linear 
program. However, the penalty parameter must approach zero in order to obtain a solution 
to the dual problem. Although this approach has been used effectively in conjunction 
with successive overrelaxation methods both on serial [10] and parallel machines [11, 13], 
we propose here the use of an augmented Lagrangian on the dual problem instead of 
an exterior penalty function in order to alleviate the twin difficulties of determining the 
threshold value of the penalty parameter required for an exact primal solution, and the 
need for the penalty parameter to approach zero in order to obtain a dual solution. The 
first proposal for using an augmented Lagrangian formulation for solving linear programs 
was made in [22]. In [18] Polyak and Tretiyakov made the remarkable discovery that after 
a finite number of steps of the augmented Lagrangian algorithm, an exact solution to 
the primal and dual linear programs is obtained. In [3] Golshtein proposed a projected 
Gauss-Seidel method in conjunction with an augmented Lagrangian formulation and gave 
computational results for linear programs with sizes up to 352 variables and 166 constraints. 
No convergence proofs of the projected Gauss-Seidel method was given in [3], nor of the 
closely related iterative method of Syrov and Churkreidze in [18]. We propose here the 
use of a projected successive overrelaxation method in conjunction with an augmented 
Lagrangian formulation. The convergence of the projected SOR method established in 
[7] is general enough to cover both a serial and a parallel implementation of the method. 
Since SOR methods are inherently serial in nature, their parallelization is not a routine 
matter. In [11, 13] two related methods were proposed for the parallelization of SOR 
methods. The more recent method [13] utilizes an unreduced relaxation factor interval of 
(0, 2) which we shall employ here with an augmented Lagrangian algorithm for the dual 
linear program. 

The paper is organized as follows. In Section 2 we give the necessary theoretical 
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background and convergence results for the proposed augmented Lagrangian method ap- 
plied to the dual linear program. In Section 3 we describe a serial SOR implementation 
of the method and establish its convergence. In Section 4 we describe our parallel SOR 
implementation and in Section 5 we present computational results for both the serial and 
parallel methods. 

We briefly describe our notation now. For a vector x in the n-dimensional real space 
R n , x+ will denote the vector in R n with components (x+) i = max {xj, 0}, i = 1, . . . , n. 
The scalar product of two vectors x and y in R n will be simply denoted by xy. For 

n 

1 < p < oo, the p-norm °f a vec tor in R n will be denoted by ||#|| p . For the 

«=i 

2-norm the subscript 2 will be dropped, i?” will denote the nonnegative orthant or the set 
of points in R n with nonnegative components, while R mXn will denote the set of all m x n 
real matrices. For A G R mXn , A T will denote the transpose, Ai will denote the ith row, 
Aij the element in row i and column j , and for I C {1, . . . , m}, J C {1, . . . <n}, Aj will 
denote the submatrix of A with rows Ai, i G I , while Ajj will denote the submatrix of A 
with elements Aij , i G /, j G J. Similarly for x G R n and It C { 1, . . . , n}, xj t will denote 
X{ , i £ It- The set {/i, J 2 , . . . , Ik} is said to be a consecutive partition of {1, . . . , n} if 
it is a partition of {1, . . . , n} such that i < j for i G It , j € It+i and I — 1, . . . , k — 1 . 
Here and throughout the symbols : = and =: denote definition of the term on the left and 
right sides of each symbol respectively. 
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2. Theoretical Background 

We consider the linear program 



(2.1) min cx subject to Ax >6, x > 0 

where c £ R n , 6 £ R m and A £ R mXn and its dual 



(2.2) max bu subject to v = —A T u -f c 

( u , t >)>0 

For simplicity we exclude trivial constraints with A,* = 0. In [8, 9] the exterior penalty 
problem associated with the dual problem (2.2) 

(2.3) max ebu — ^ ||A r u 4- v — oil 2 

(«,v)>o 2 

was solved by an SOR procedure for a sufficiently small value of the penalty parameter 
p to obtain (u(e), v(e)) . The unique least 2-norm solution x of the linear program (2.1) 
was obtained by using the equation 

(2.4) s(e) = ~ (A T u(e) + v(e) - c) 

which relates an optimal solution (u(e), u(e:)) of the dual penalty problem (2.3) and the 
unique solution x(e ) of the corresponding quadratic primal problem [5] 



(2.5) min cx + ^ xx subject to Ax >6, x > 0 

In particular it follows [14] that the least 2-norm solution x of the linear program (2.1) is 
related to x(e) of (2.4) by 



( 2 . 6 ) 



x = x(e) for e £ (0, e] for some e > 0 



Thus the penalty parameter e of the dual penalty problem (2.3) is the perturbation param- 
eter of the perturbed primal problem (2.5). In order to avoid possible difficulties associated 
with determining the threshold value e we consider instead of the exterior penalty problem 
(2.3) the augmented Lagrangian associated with the dual linear program (2.2) 



L(u, v, x , 7) := bu - || A t u -ft; — c|| 2 — x(A T u -f v — c) 

27 



(2.7) 
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It is a standard result [ 6 , 2 , 19] that for any 7 > 0, a primal-dual solution (x, u, v ) of 
(2.1)-(2.2) is equivalent to a stationary point of the following saddlepoint problem of (2.7): 
Find an (x, u, v) £ R n x R™ x jR+ such that for all x £ R n and all (u, v) £ R £ x R " , 

( 2 . 8 ) L(u, v , x, 7 ) < L(u, 0 , x, 7 ) < L(u, v, x, 7 ) 

The standard augmented Lagrangian algorithm [ 2 , 19] consists of a maximization step in 
the ( u , v) space il™ x followed by an unconstrained gradient descent step in the x 
space R n . In particular we have the following. 



2.1 Augmented Lagrangian Algorithm 



(2.9) 



Start with any x° £ R n . Having x* determine x * +1 as follows 
( (a) L(u\ v\x t ,y t ) = max L(u,v,x\ 7 *) 

1 («,v)>0 

(b) x ,+1 = x* — ~ V x L(u\ v\ x‘, 7 *) = x* + i(A T u* + w 1 — c) 

7* 7* 



where { 7 *} is any bounded sequence of positive numbers. 

For this iterative linear programming algorithm Polyak and Tretiyakov [18] have given 
the following important finite convergence theorem. 



2.2 Augmented Lagrangian Algorithm Finite Termination Theorem [18] 

For any bounded positive sequence { 7 *} and x° £ R n , Algorithm 2.1 is finite, that is 
there exists an integer k such that (x*, u*, v k ) solve the dual linear programs ( 2 . 1 )-( 2 . 2 ). 
Furthermore for each x° there exists a 7 > 0 such that for 0 < 7 0 < 7 , the method will 
terminate in one step, that is (x 1 ,^ 1 , v 1 ) will solve the dual linear programs ( 2 . 1 )-( 2 . 2 ). 

Note that in the above theorem, two exact maximizations over the (u, v ) space R™ x 
R+ are required in order to obtain (x ,+1 , u ,+1 , v ,+1 ) from x*. 

We note that since by duality theory [5] 



(2.10) max L(u, v,x, 7 ) = min {cz + j- \\z — x || 2 \Az > 6 , z > 0 } =: <j>(x) 

(u,v)>0 z 2 I 

the Augmented Lagrangian Algorithm 2.1 is equivalent to the following gradient method 
applied to the proximal point function <f>(x) 



( 2 . 11 ) 



x * +1 = X* rV^(x') = Prox (x‘) 

7* 

where Prox (x*) is the solution of 
min {cz -f- \\ z ~ x% || 2 ^ z > 0 ) 
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Bertsekas [1] and Rockafellar [21] also obtain finite termination for (2.11) from proximal 
point theory considerations for the linear programming case. 

With this background we are prepared now to state and prove the convergence of our 
serial and parallel SOR algorithms. 
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3. Serial Successive Overrelaxation Algorithm 

The proposed serial algorithm consists of applying the projected SOR method of [7] to 
the maximization step (2.9a) of the Augmented Lagrangian Algorithm 2.1 for a decreasing 
sequence of positive numbers { 7 *}. It follows from the Finite Termination Theorem 2.2 
and a theorem of Pang [16, Theorem 3.1], that for any x * the projected SOR method 
will generate a sequence of points converging to an x x+l that solves the primal linear 
program ( 2 . 1 ), provided that 7 * is sufficiently small. There are no easily implementable 
and theoretically justifiable ways of determining how to choose 7 * sufficiently small [ 20 , 4 ] , 
however we shall prescribe some computationally effective ways for doing that. 

The serial projected SOR method has been proposed [7, 9] for solving the quadratic 
minimization problem 

(3.1) min 0(z) := min — zMz + qz 

v z>0 w z>0 2 



where M £ R kxk [ s symmetric and positive semidefinite. This is precisely problem (2.9a) 
of Algorithm 2.1 if we make the identifications 



(3.2) 



M: = 



7’ 



[A? 1} 



r ^ - 7) ■ - 6 i 




2 := 



u 

V 



The projected SOR algorithm consists of the following. 

( - (*j 4 . ...4)) 

I u € ( 0 , 2 ), j = 



(3.3) 



More specifically for 0(z) = j zMz + qz we have the following. 

3.1 Serial SOR Algorithm for min 5 - zMz + qz 

®>0 A 

Choose z° £ R+, u: £ ( 0 , 2 ). Having z* compute z t+1 as follows 

z j +1 = (*} - ( y, M n z l +1 + Y M > tzt t + 

( 3 - 4 ) V J + 



j = 1, ... ,k 
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We are ready to state and establish the convergence of our augmented Lagrangian 
serial SOR algorithm. 

3.2 Augmented Lagrangian Serial SOR Algorithm 

Let {7*} l 7 for some 7 > 0, let {<$* } j 0 and let x° £ R n . Having x l determine 
x* +1 as follows: 

(a) Apply the Serial SOR Algorithm 3.1 to solve (2.9a) with the identifications (3.2), and 
let (w l (£), v f (t)) be the i iterate of this SOR algorithm. Stop if for some t — t l the 
following inequality is satisfied. 

I u‘(*)V u L(tP(t), t /(*), x\ 7') I + I </(*)V v L(u‘(*), v\t) 9 x\ 7*) I 

(3.5) 

+ ||(V u Z,(tt'(<), «•■(<), 70+11 + ||(v K L( W *(t), v\t), x\ 7 < )) + || < F 

(b) Set x* +1 = x x (t l ) where 

(3.6) x\t) := x i + X(A T u i (t) + «*(<) - c) 

7 * 



3.3 Augmented Lagrangian Serial SOR Convergence Theorem 

Let {7*} 1 7 > 0 be a sufficiently rapidly decreasing sequence of positive numbers 
and 7 sufficiently small. Then either 

(a) For some integer k , the sequence {x*(tf)} converges to an x k that solves the linear 
program (2.1), or 

(b) For each subsequence of {(x*, u l (t l ), u*(t*))} converging to some (x,w, v), the cor- 
responding subsequence {x* +1 = x*(t*)} converges to an x such that x solves the 
linear program (2.1). If x = x, then (w, v) solves the dual linear program (2.2). 

Proof Either the inequality (3.5) of the algorithm is satisfied at each iteration i for some 
t l or not. Accordingly we have the alternatives (b) and (a) below respectively. 

(a) For some iteration i = k the inequality (3.5) is never satisfied. Hence by Pang’s 
Theorem 3.1 [16], since L(u , v , x fc , 7*) is by duality theory bounded above for (u, v) > 
0 by cx + 7*||x — x k \\ 2 for any x > 0 such that Ax > b , it follows that the sequence 



(3.7) 



{**(<)) := { 



r k . A T u k (t) + v k (t)~c 
X \ L 



< = 0 , 1 , 2 ,..., 




Ill 



where t is the SOR iteration index, converges to a vector x k defined by 



(3.8) 



2 * + 
7* 



for some which solves max L(u,v,x k , 7 *). Note however that {tt*(t), v*(t)} 

need not converge to ( u k ,v k ). If 7 * is sufficiently small (and this is what is meant by 
requiring that { 7 *} decreases sufficiently rapidly) it follows by Theorem 2.2 that x * +1 = 
x*, is a solution of the primal linear program (2.1). (Note that (u k ,v k ) need not be a 
solution of the dual linear program (2.2). To obtain such a dual optimal ( u k , v k ) we need 
to solve min L((u,v,x k , 7 *) exactly. See Corollary 3.5 below.) 

(«,v)>0 



(b) If inequality (3.5) holds for each i for some t*, then since {£*} l 0 and { 7 *} j 
7 > 0 , we have that for any subsequence of {(x , ,u*(t*), v*(t*))} converging to some 
(x,u,v), the point ( u,v ) solves the problem min L(tt,t;,x, 7 ) (because { 6 *} J, 0 ), 

(tt,v)>0 

and moreover the corresponding subsequence {x ,+1 } defined by (3.6) and part (b) of 
Algorithm 3.2 converges to an x defined by 



(3.9) 



X 



X + 



A t u + v — c 



7 



If * is sifficiertly small, then it follows by Theorem 2.2 that x solves the linear program 
( 2 . 1 ). If in addition x = x, then (u,v) is feasible for the dual linear program ( 2 . 2 ) and is 
also optimal because bu = cx. I 

It is useful to point out that when the sequence {(u*(£), u*(t))} of Algorithm 3.2 is 
bounded then it has an accumulation point and by Theorem 2.1 [7], each such accumulation 
point satisfies the optimality conditions for (2.9a). Consequently for each i inequality (3.5) 
of Algorithm 3.2 is satisfied after a finite number of steps of part (a) of the algorithm. 
However by Theorem 2(ii) of [9] we have that if the linear program ( 2 . 1 ) satisfies the Slater 
constraint qualification, then the sequence {(tt*(f), u‘(t))} is indeed bounded. Therefore 
we have the following. 



3.4 Augmented Lagrangian Serial SOR Convergence Corollary 

Let the linear program ( 2 . 1 ) satisfy a Slater constraint qualification, that is Ax > 6 
for some x > 0 . Then Theorem 3.3 holds with outcome (b). 
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Another useful observation follows from the fact [18, Lemma 1], [19] that minimizing 
the augmented Lagrangian L(u, v, x, 7) with an optimal value of x and any 7 > 0 gives a 
solution to the dual linear program (2.2). Hence we have the following. 

3.5 Dual LP SOR Solution Corollary 

Under the assumptions of Theorem 3.3 a solution to the dual linear program (2.2) can 
be obtained for either outcome (a) or (b) of Theorem 3.3 by solving respectively 

(a) min 

(u,v)>0 

or 

(b) min L(u,v,x,y) 

(u,v)>0 

We note immediately that we have left open the procedure by which the sequence 
{7*} is decreased. This is an inherent theoretical difficulty that arises when using inexact 
minimization in the subproblems of proximal point or augmented Lagrangian algorithms. 
Thus the approximate minimization criteria of [21, Criteria A, B , A', B'] are not imple- 
mentable for our problem, while the assumptions of [2, Section 2.5] are not verifiable for our 
problem. Computationally we have overcome this difficulty by using the following scheme, 
often used for updating the penalty parameter in augmented Lagrangian algorithm 



(3.10) 



7 



«+i = 



7* if ||x ,+1 — ar* || < n\\x l - x x 1 1|, 0 < fi < 1 
1/7* otherwise, 0 < v < 1 



This scheme works effectively for the solution of very large sparse linear programs as our 
computational results indicate. 
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4. Parallel Successive Overrelaxation Algorithm 

The key to our parallel SOR algorithm is the use of the parallel gradient projection 
SOR (GP-SOR) that we proposed in [13] for the solution of (3.1) and which we outline 
below. Partition the matrix M of (3.1) into r contiguous horizontal blocks as follows: 



(4.1) 



M =: 



■M Ix I 

M h 



lM Ir J 



where the blocks Mj i correspond to the variables , and {Ij, / 2 , . . . , J r ) is a consecutive 
partition of {1, 2, . . . , k } . Now partition Mj j as follows 



(4.2) 



Mi j =: [Mijij Mj.j. ] 



where Ij is the complement of Ij in {1,2,..., A:} . Thus Mi j i j is a principal square 
submatrix of M with elements M st , s £ Ij and t £ Ij. We further partition Mj j i j into 
the sum of its strictly lower triangular part Li.j. , its diagonal part Di j i j and its strictly 
upper triangle part Cf/.j. as follows 



(4.3) 



Mijij -■ L IjIj + D IjIj + 17/./. 



Now define a block diagonal matrix K as follows 



(4.4) 



K: = 



L hh 






Li r i r J 



where each Li j i j is the strictly lower triangular part of Mi j i j . An SOR algorithm can 
now be performed for each row block Ij, j = 1, . . . , r, simultaneously, that is in parallel. 
Note that the block diagonal matrix K replaces the traditional strictly lower triangular 
matrix of the serial SOR. Specifically we have the following. 



4.1 Parallel GP-SOR Algorithm for (3.1) 

Let • • • ,/ r } be a consecutive partition of {1,2,..., A;}, let E be a positive 

diagonal matrix in R kxk and let z° > 0. For i = 0, 1,2, ... , do the following 
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Direction Generation Define the direction 

/ p/i( z< ) - 4, 

( 4 . 5 ) <* = ***)-**: = ; 

\P/r(^)-4 r 

such that p(z % ) satisfies 

( 4 . 6 ) Pij(z') = (z\. - wEijtj {M IjZ ' + qi . + Lijjjipi^z') - *}.))) + , 3 = -. r 

where u> > 0 is chosen such that for some v > 0 

( 4 . 7 ) ((uEijij )" 1 + L JjJ .)zi i > i/ll^/,. ||\ Vz/., j = 1, ..., r 

Stop if d‘ = 0, else continue. 

Stepsize Generation z ,+1 = z* 4- A* cl* 
where 



(4.8) /(** + A'd') = nun {/(*•’ + Ad 4 )!* 4 + Ad‘ > 0} 



4.2 Remark The principal part of this algorithm consists of the direction generation part 
(4.6), which can be performed in parallel on r processors. Once this is done the stepsize 
generation (4.8) is performed and the new value z'+ l is shared between the r processors. 

The following convergence results were derived in [13] for Algorithm 4.1. 

4.3 Theorem (Convergence of the Parallel GP-SOR Algorithm) Let M be symmetric 
and positive semidefinite. Either the sequence {z‘} generated by the Parallel GP-SOR 
Algorithm 4.1 terminates at a solution of (3.1) or each of its accumulation points solves 
(3.1). 

4.4 Corollary (Parallel GP-SOR special cases) Condition 4.7 of Algorithm 4.1 holds 
under either of the following two assumptions: 



0 < cj < min 

,r 



min 



2 






(4.9) 
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(4.10) 0 < w < 2, E = D- 1 and M is positive semidefinite 

Our parallel augmented Lagrangian method consists of replacing the Serial SOR Al- 
gorithm 3.1 by the Parallel GP-SOR Algorithm 4.1 in Algorithm 3.2 with option (4.10) 
for the choice of to and E . We formally state the algorithm below. 

4.5 Augmented Lagrangian Parallel SOR Algorithm 

Identical to Algorithm 3.2 except that the Serial SOR Algorithm 3.1 in Algorithm 3.2 
is replaced by the Parallel GP-SOR Algorithm 4.1 with condition (4.10) above in force. 
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5. Computational Results 

Our algorithms were tested on random linear programs which were generated as fol- 
lows. First the matrix A of the linear program (2.1) was generated. Each of its nonzero 
elements was a random number generated by a uniform distribution on the interval [-100, 
100]. The number of nonzero elements in each row was in accordance to a prescribed 
density and the random position of each nonzero element was determined according to 
a uniform distribution on the column indices of the matrix. Next a primal-dual solution 
vector (x,u) was generated from a uniform distribution on the interval [0, 10] with 80% 
of the components being nonzero. Finally the vectors b and c were chosen so that ( x,u ) 
is optimal. 

In Figure 1 we give a summary of our computational results for 6 problems with the 
number of constraints varying between 25,000 and 125,000 and the number of variables 
varying between 100,000 and 500,000. All tests were performed on a MicroVax II with 16 
megabytes of memory and an expanded disk swap space. We are not aware of any other 
linear programming software that can handle problems of the size that we have attempted 
on a comparable machine. MINOS [15], a state-of-the-art pivotal linear programming 
package, cannot handle any of the problems listed in Figure 1 because they are too big for 
the machine memory using the MINOS configuration. The largest problem attempted on 
the MicroVax II with MINOS was a problem with 5000 constraints, 20,000 variables and 
a matrix density of 0.2% with about 200,000 nonzero elements. MINOS was used with 
the standard partial pricing and scaling options. After 3150 iterations and 49 hours 54 
minutes of machine time, the point was infeasible and the objective function was in error 
by 59% of the exact minimum. By comparison our Algorithm 3.2 solved the same problem 
in 1 hour and 4 seconds with a primal-dual objective function accuracy of 7 figures, and 
relative accuracy of not less than 10 -9 as defined in Figure 1. 

The Parallel SOR Algorithm 4.5 was implemented on the Sequent Balance 21000, a 
multiprocessor that incorporates eight NS32032 processors running at 10MHz, each with 
a floating point unit, memory management and an 8-kbyte cache sharing a global memory 
via a 32-bit wide pipelined bus. The machine has 8-Mbytes of physical memory. The 
operating system DYNIX, is a version of Berkeley 4.2 bsd unix. The computational results 
are depicted in Figures 2 and 3. 
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Figure 2 shows the total computing time versus number of processors for four different 
densities: d— 1, 2, 7 and 10 percent for a linear program with 1000 constraints and 4000 
variables. All problems were solved to a 7-figure accuracy of the primal-dual objective 
function and relative accuracy better than 10~ 7 as defined in Figure 1. We observe that 
the optimal number of processors, that is the one that solves the problem in minimum 
total time, increases with density as expected. This number is 3 for 1% and 2% densities, 
6 for 7% density, and 7 or more for 10% density. This means that for denser problems, a 
larger number of processors is needed in order to arrive at the shortest solution time. This 
also means that for denser problems, the communication cost does not become a dominant 
and hence prohibitive factor until a larger number of processors are used. 

In Figure 3 we show results for the case with 10,000 constraints and 40,000 variables, 
with density of 0.35% and about 1,400,000 nonzero elements. To our knowledge this is one 
of the largest linear programs solved on this relatively modest sized multiprocessor. One 
of the reasons that we were able to solve larger problems on the MicroVax II, is that the 
latter had twice the total memory size of the Balance 21000 and furthermore, the MicroVax 
was essentially a single-user machine dedicated to the serial SOR algorithm. The optimal 
number of processors for the low-density case shown in Figure 3 is 4. Just as in the cases 
of Figure 2, the optimal number of processors should increase with problem density. 

We conclude with some observations on the speedup efficiency of our Parallel SOR 
Algorithm 4.5. We define the speedup efficiency E(r) as the ratio of the actual to the 
theoretical speedup of the algorithm using r processors instead of 1 processor, thus 

< 5I > 

where T(r) is the total time for solving a given problem using r parallel processors. 
Figure 4 shows the speedup efficiencies for a typical case of a linear program with 1000 
constraints, 4000 variables and 2% density. The reason why some efficiencies are over 
100% was pointed out in [13]. The explanation is that our Parallel SOR Algorithm 4.5 
changes with the number of processors used, because the matrix K defined by (4.4) changes 
with the number of blocks into which M is divided. Thus we are not comparing identical 
algorithms when we evaluate the ration T(l)/rT(r) of (5.1). Nevertheless the expression is 
a valid measure of efficiency in the sense of comparing the theoretical reduced time T(l)/r 
to the observed time T(r) for an algorithm with a variable K that depends on the partition 
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of M. If the matrix K is held fixed for r = 1 and r > 1, then we obtain efficiencies for 
identical algorithms, and they would all be less than 100%. This was demonstrated in 
[13]. Nevertheless the present efficiencies of over 100% are indeed very encouraging and 
also give the additional and somewhat surprising result that a serial implementation of 
our r -block Parallel SOR Algorithm 4.5 on a single machine, will give for some r a better 
computing time than the single block Serial SOR Algorithm 3.2. For the specific case of 
Figure 4, a serial implementation of the r -block parallel SOR with r=2, 3, 4 and 5 will 
be faster than the single block serial SOR. 
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Prob. 

No. 


No. 

constr. 

mxlO -3 


No. 

var. 

nx 10“ 3 


Nonzero 
elements 
per row 


Total 

Nonzero 
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xl0“ 5 


Iter 


Time 

hr:min 


Fig. Accur. 
Obj. Func. 
P D 


1 


Logio Rel. 
Accur. 

2 3 


4 
















(a) 


(b) 


(c) 


(d) 


(«) 


(f) 


1 


25 


100 


20 


5 


309 


2:31 


7 


7 


-7 


-10 


-9 


-10 


2 


30 


120 


25 


7.5 


392 


4:34 


7 


7 


-10 


-14 


-12 


-14 


3 


40 


160 


20 


8 


363 


4:56 


7 


7 


-9 


-11 


-7 


-8 


4 


50 


200 


16 


8 


525 


7:47 


7 


7 


-9 


-8 


-9 


-8 


5 


100 


400 


12 


12 


967 


22:35 


5 


7 


-3 


-5 


-5 


-7 


6 


125 


500 


9 


11.25 


3100 


71:40 


6 


7 


-3 


-5 


-5 


-7 



(a) Number of correct figures in primal objective 

(b) Number of correct figures in dual objective 

(c) ||(-A* + 6) + || oo /||(6) + || oo (Relative Accuracy) 

(d) ||(A T u-c) + || oo /||(-c) + || oo (Relative Accuracy) 

(e) | cx — cx\/\cx + cx |, x: exact, x: computed (Relative Accuracy) 

(f) |6u — 6u|/|6u + 6u|, u: exact, u : computed (Relative Accuracy) 



Fig. 1. MICRO VAX II: Serial SOR Algorithm 3.2 test results. 






Time in Seconds 
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Total Time (d = 1%. m = 1000, n = 4000) Total Time (d = 2%. m = 1000, n = 4000) 





Total Time (d = 7%, m = 1000, n = 4000) Total Time (d = 10%, m = 1000, n = 4000) 





Fig. 2. BALANCE 21000: Total time for Parallel SOR Algorithm 4.5 to solve linear program 
versus number of processors for various densities d. (Average of 4 randomly generated 
cases with 1000 constraints and 4000 variables.) 




121 



Total Time (d = 0 35%, m = 10000. n = 40000) 




Fig. 3. BALANCE 21000: Total time for Parallel SOR Algorithm 4.5 to solve linear program 
versus number of processors. (<i=density, m=number of constraints, n=number of 
variables.) 






Density d = 2%, 


m = 1000, n = 4000 


No. 


Time Sec. 


Speedup 
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T(r) 


Efficiency 


r 




E{r) = T(l)/rT(r) 


1 


3946 




2 


709 


278% 


3 


638 


206% 


4 


716 


138% 


5 


711 


111% 


6 


840 


78% 


7 


850 


66% 



Fig. 4. BALANCE 21000: Speedup efficiency E(r) for the Parallel SOR Algorithm 4.5 for 
an LP with 1000 constraints and 4000 variables and 2% density. 
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EMP: An Expert System for Mathematical Programming 

- Klaus Schittkowski - 



Abstract : 

The paper describes a software system that supports model 
building and numerical solution of various mathematical pro- 
gramming problems, e.g. of nonlinear programming, data fit- 
ting, min-max programming, multicriteria optimization, non- 
smooth optimization, quadratic programming, or linear pro- 
gramming, subject to linear or nonlinear constraints. All 
actions of the system are controlled by menues which inform 
the user how to generate, solve, edit, delete or list a 
problem, for example. Nonlinear problem functions must be 
defined by sequences of FORTRAN statements assigning a nu- 
merical value to a user-provided name. The system writes a 
complete FORTRAN source program, which is linked and executed 
automatically. All problem data and numerical results are 
stored in a data base and are easily retrieved for further 
processing. For each nonlinear problem, at least two diffe- 
rent mathematical algorithms are available. The system is 
capable to learn, i.e. to improve its own knowledge on the 
success of the algorithms, and to perform a rule-based error 
analysis in case of nonsuccessful termination. 



Mathemat isches Institut 
Universitaet Bayreuth 
D - 8700 Bayreuth 




126 



1. General information on EMP 



The development of efficient and reliable algorithms for 
solving practical optimization problems was one of the main 
research areas of the last 20 years in mathematical 
programming. In the present situation a significant amount of 
numerical experience and knowledge on the performance of 
computer programs is available, in particular in linear 
programming, quadratic programming, least squares optimi- 
zation, and smooth nonlinear optimization. Only in linear 
programming, we observe major additional attempts to facili- 
tate model building, program execution and report writing for 
those who have to solve practical application problems. But 
based on the previous comment, we have to be aware that also 
in other areas, mathematical programming codes will be in the 
hands of non-specialists or occasional users, who have diffi- 
culties (or do not possess the time) to find a suitable algo- 
rithm, to transform the problem into the required form, to 
understand the documentation and to execute the FORTRAN code. 

Expert systems are computer programs that embody the 
expertise of specialists in some problem area and were 
developed in many different disciplines in the last 15 years, 
e.g. MYCIN for medical consultation, cf. Shortliffe (1976), 
PROSPECTOR for mineral exploration, cf. Duda, Gasching and 
Hart (1979), or HEARSAY II for speach recognition, cf. Lesser 
and Erman (1977). Besides of a suitable knowledge base, an 
expert system contains an inference mechanism to perform the 
reasoning process and to construct inference paths. Its basic 
characteristics can be summarized as follows, see Hayes-Roth, 
Waterman and Lenat (1982) for additional information: 

1. The system must perform well by applying expert rules in 
an efficient manner to achieve acceptable solutions. 

2. Knowledge is to be represented in a symbolic way for 
realizing the internal reasoning process. 

3. The system should possess some knowledge of the basic 
principles of its domain of interest, e.g. to provide 
additional explanations when expert rules fail. 

4. It must be able to convert the problem description in lay 
terms into an internal representation convenient for 
processing with its expert rules. 

5. The expert system must be designed to solve domain 
specific real world problems. 

EMP is an interactive programming system that supports 
model building, numerical solution and data processing of 
constrained mathematical programming problems. Various 
options are available in EMP to facilitate the formulation of 
problem functions. The objective function e.g. may be a 
linear or quadratic function, a data fitting function, a sum 
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or maximum of functions, or a general function without a 
structure that could be exploited. More precisely the 
following mathematical models are available for facilitating 
the formulation of objective or constraint functions and 
exploiting special structures mathematically whenever 
possible : 

Linear programming 
Linear regression 
Quadratic programming 
Nonlinear Ll-data fitting 

Nonlinear L2- or least squares data fitting 
Nonlinear maximum-norm data fitting 
Multicriteria optimization 
Min-max optimization 
Non-smooth optimization 
General nonlinear programming 

All problems may have bounds for the variables and linear 
or nonlinear constraints. In both cases it is possible to 
proceed from two-sided bounds for the restrictions. Data 
fitting problems are either composed of a sequence of 

arbitrary nonlinear functions or of one model function, where 
the experimental data are provided separately. Further 

mathematical models are in preparation, e.g. optimal control. 

For most optimization problems, many different algorithms 
are available. The present version of EMP contains 
mathematical methods of the following type: 

Sequential quadratic programming method for nonlinear 
programming 

Bundle method for non-smooth or ill-conditioned problems 
Levenberg-Marqurdt , Gauss-Newton, Newton, and quasi- 
Newton methods for nonlinear least squares problems 
Ellipsoid method for smooth and non-smooth optimization 
Dual and primal methods for quadratic programming 
EMP includes program generators for codes of the frequently 
used optimization libraries NAG and IMSL, and the algorithm 
base is extended in future step by step. 

For objective function and constraints, the input of 
quadratic or linear functions reduces to definition of some 
vectors and matrices, respectively, where sparsity can be 
exploited. Gradients of nonlinear and nonquadratic functions 
are approximated numerically, but can also be provided by the 
user in analytical form. The input of ’simple* objective or 
constraint functions is facilitated, if they differ at most 
by an index. 

Only the problem relevant data need to be provided by a 
user in an interactive way. General functions must be defined 
by sequences of FORTRAN statements addressing a numerical 
value to a user provided function name. All generated 
problems are stored in form of a data base system, so that 
they are easily retrieved, modified, or deleted on request. 
EMP proposes a suitable mathematical algorithm and writes a 
complete FORTRAN source program. The system executes this 
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program and stores the numerical results in the data base, so 
that they are available for further processing. Since 
individual names for functions and variables can be provided 
by a user, it is possible to get a problem dependable output 
of the achieved solution. 

The user will be asked whether he wants to link the 
generated FORTRAN program with some of his own files or 
whether he wants to insert additional subroutines, decla- 
ration and executable statements to formulate the problem. It 
is possible to generate the same programs automatically, that 
must be generated by ’hand* otherwise. 

In various ways the system is capable to learn and to 
store its own experience on the success of solution attempts. 
The proposals offered by EMP, will therefore become better 
and better with increasing knowledge on the numerical 
structure of the user provided models. A rule-based failure 
analysis explains some reasons for possible false termi- 
nations and proposes remedies to overcome numerical difficul- 
ties . 

All actions of EMP are controlled by self-explained com- 
mands which are displayed in form of menues . Step by step the 
user will be informed how to supply new data. Whenever 
problem data are generated or altered, the corresponding 
information will be saved on a user provided file. Besides 
commands to generate, solve or edit a problem, there are 
others to transfer data from one problem to another, to 
delete a problem, to sort problems, to get a report on 
problem or solution data, to halt the system and to get some 
information on the system, the mathematical models and the 
available algorithms. It is even possible to insert arbitrary 
operating system commands without leaving EMP. 

The main intention of EMP is to prevent the organisational 
’ballast* otherwise required to solve a nonlinear programming 
problem with a special algorithm. Once the system is impl- 
emented, it is not necessary 

to define a special file for each problem to be solved, 
to select a suitable mathematical algorithm, 
to read any documentation of the used mathematical 
programming algorithms, 

to write long lists of declaration statements, e.g. for 
dimensioning auxiliary arrays required by the algorithm, 
or to call the mathematical programming code with a long 
list of parameters that are to be defined initially, 
to provide the problem functions and their gradients in a 
special form required by the mathematical programming 
algorithm , 

to define an individual output file for each set of 
results , 

to interprete the results for a decision maker. 

Thus the domain of application of EMP is summarized as 
follows : 
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(i) Programming neighbourhood for developing a first 
executable program version solving a specific practical 
problem (or class of problems). 

(ii) Investigation of different model variants fitting best 
to a given real world situation. 

(iii) Testing certain types or modifications of mathematical 
programming algorithms for solving a class of problems. 

(iv) Collecting numerical experience on solution methods for 
optimization problems. 

(v) Teaching students on model building (e.g. structural 
optimization courses in engineering science) or on 
numerical behavior of optimization algorithms (e.g. 
optimization courses in mathematics). 

EMP allows a user to concentrate all his efforts on the 
problem he wants to solve and takes over the additional work 
to select a suitable algorithm and to organize the data, the 
execution of the problem functions and the program structure. 
It should be possible to solve optimization problems of the 
class under consideration within a minimal fraction of time 
needed otherwise. 

EMP is implemented in SUSY, cf. Schittkowski (1987), a 
language that was designed entirely for the generation of 
interactive problem solving systems like EMP with the 
additional feature to process heuristic knowledge and 
uncertainties. A corresponding interpreter was implemented by 
the author and is available on request together with EMP, at 
present only for VAX/VMS computing environments. A UNIX- 
version is in preparation. 

The presented system to facilitate model building and 
numerical solutionof mathematical programming problems, is 
based on the host language FORTRAN and it is assumed, that a 
user is familar with this language. A completely different 
approach is obtained by designing a specific model language 
as known from linear programming. An extension to nonlinear 
problems was implemented by Brooke, Drud, and Meeraus (1984) 
at the World Bank, and the resulting system is used at 
present to develop and solve operations research models. 
However a model language must be learned and the application 
is restricted to those models that fit into the system. EMP 
was designed mainly for engineering applications, where we 
need a very flexible input procedure, since possible 
application problems are highly nonlinear, and where we can 
suppose that users are famil&r with FORTRAN. Since the 
generated programs can be used outside of the system, one 
could consider EMP also as a software development tool. 

Although EMP can be used in form of a ’stand-alone 1 
product, additional efforts can be undertaken to extend its 
knowledge base and to improve its intelligence. In particular 
a user could alter existing models and to define new ones. In 
addition the system is implemented in a way to allow an 
extension of the existing algorithm base very easily, i.e. to 
scratch or add new algorithms for an existing model. The 
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usage of the special system language SUSY allows an efficient 
alteration and adaption of the whole program to another 
situation. A complete documentation of SUSY is included in 
the system and it is possible to extract individual chapters 
interactively . 
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2. Mathematical models 



EMP is intended to formulate and solve nonlinear program- 
ming problems of the following general structure: 



minimize F(x) 

subject to G(l,x) = 0 

G ( me , x ) = 0 

G(me+1 ,x) >= 0 

G(m,x) >= 0 

xl <= x <= xu 



The used standard notation can be replaced by actual names 
for problem functions or variables and has the following 
meaning : 

n: Number of variables 

me: Number of equality constraints 

m: Number of constraints without lower and upper 

bounds on the variables 

x: Vector of n independent design variables. 

F(x): Real-valued objective function which is to be 

minimized. It is assumed that F depends on the n 
variables x(l), ... ,x(n) and that F is continuous 

at least in the rectangle defined by the lower and 
upper bounds on the variables, or is composed of 
continuous functions. 

G(l), ... ,G(m): Set of m real-valued constraint 

functions defining the feasible region of the 
problem. They are assumed to be continuous with 
respect to the variables x(l), ... ,x(n) at least 

in the rectangle definied by the bounds on the 
variables . 

xl,xu: Two n-dimensional vectors defining lower and upper 

bounds for the variables. 



Alternatively it is possible to proceed from two-sided 
bounds for the constraints, i.e. from a set of constraints of 
the form 







G ( 1 , x ) = 


b ( 1 ) , 






G(me,x) = 


b(me) , 


b ( me+ 1 ) 


< = 


G(me+l,x) <= 


b(m+l ) , 


b(m) 


< = 


G(m,x) <= 


b(m+m-me) 


EMP transforms 


the 


above restrictions into a suitable form 



required by the numerical algorithms. The constant bound b is 
defined separately from the m constraint functions 
G ( 1 , x ) , . . . ,G(m,x) . 
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EMP is also capable to solve linear and nonlinear systems 
determined by equality and inequality conditions, respec- 
tively. In this case, only the constraints are declared by 
the user and EMP inserts the objective function 

F ( x ) = x’*x , 

so that a feasible solution with minimal Euclidean norm is 
calculated. Note that x* denotes the transpose of the vector 

x . 



To facilitate input and output of problem functions and to 
exploit special mathematical structures by the choice of a 
suitable algorithm, some options are available for providing 
the functions F and G(l), ... ,G(m) . 



a) Objective function: 



Several options are available for formulating the 
objective function and for facilitating its input: 

Linear function, i.e. 

F ( x ) = C’*x 

with an n - vector C. Only the coefficients of C need to 
be determined. 

Linear least squares function, i.e. 

F ( x ) = .5*1 |C*x - D | | 

where | | . I ) denotes the Euclidean norm. C is an 1 by n 

- matrix, D an 1 vector. Only C and D need to be 

defined in this case. Note that all displayed objective 
function values show the squared norm without the 
constant term . 5*d**d. 

Quadratic function, i.e. 

F ( x ) = . 5*x * *C*x + x'*D 

where C is an n by n - matrix, and D an n - vector. Only 
C and D must be defined in this case. 

Ll-data fitting function, i.e. 

F ( x ) = |H1 (x) | + ... + |Hl(x) | 

The individual functions HI,..., HI are assumed to be 
continuously differentiable. Alternatively each function 
can be considered as a result of a parameter estimation 
experiment subject to a model function H(t,x), so that 



Hi ( x ) 



w(i)*(H(t(i) , x ) - y ( i ) ) 
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for i=l,...,l, where w(i) denotes the weight, t(i) the 
independent model data, e.g. observation time, and y(i) 
the dependent or observed data of the i-th experiment, 
i=l,...,l. It is allowed that t defines a vector of 
different independent model data. Besides of these data, 
only the model function H(t,x) needs to be declared, 
which depends on a real variable t and a vector x of the 
n independent variables. H should be continuously 
differentiable . 

L2-data fitting or nonlinear least squares function, i.e. 

F ( x ) = HI ( x ) * * 2 + . . . + HI ( x ) * * 2 

The 1 observation functions HI, ... ,H1 are continuously 

differentiable and are declared individually instead of 
the composed function F. As above each Hi can be replaced 
by a parameter estimation experiment, i.e. by a term of 
the form w( i ) * ( H( t ( i ) , x ) - y(i)). 

Maximum-norm data fitting, i.e. 

F(x) = maximum { | HI ( x ) | , . . . , | HI ( x ) | } , 

where all individual functions are continuously differen- 
tiable. Again each function can be considered as the 
result of a parameter estimation experiment subject to a 
model function H(t,x). 

Maximum of functions, i.e. 

F(x) = maximum {HI ( x ),..., HI ( x ) } 

The individual functions HI,..., HI are continuously 
differentiable and could be realizations of a parameter 
estimation experiment subject to a model function H(t,x). 

Vector of functions, i.e. 

F(x) = ( HI ( x ) , ... , HI ( x ) ) ' 

Each individual function must be continuously differenti- 
able. In an interactive way, the user will be asked to 
define the way how the multicriteria decision is to be 
performed. Four different attempts are available: 

1. Sum of weighted functions 

2. Sum of weighted functions with additional bounds on 
objective functions 

3. Selecting one function, suppressing the other ones 

4. Combination of ideal solution and the center of best 
value points 

The corresponding weights are definied interactively when 
executing the FORTRAN-code , and can be modified during 
the same session. 
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Sum of functions, i.e. 

F ( x ) = HI ( x ) + . . . + HI ( x ) 

As above, only the functions HI, ... ,H1 are to be 

defined by the user and these functions should be 
continuously differentiable. 

Sum of simple functions, i.e. 

F ( x ) = H ( 1 , x ) + . . . + H ( 1 , x ) 

Here it is assumed that each individual function H(i,x), 
i=l,...,l, possesses the same mathematical structure and 
depends only on an index i and the vector x of the 
decision variables. Again the function H(i,x) is to be 
continuously differentiable. 

- General function, i.e. it is supposed that no specific 
structure is available that could be exploited. 

b) Constraints: 

The restrictions are either assumed to be linear or of a 
general nonlinear type: 

No constraints, i.e. m is equal to zero. Since there is 
no need to provide special purpose algorithms in this 
case, the resulting problems are solved by the available 
constrained mathematical programming algorithms. 

Linear constraints, i.e. 

G ( j , x ) = x* *a( j ) + b( j ) 

The n - vector a(j) is the j-th row of an m by n - matrix 
A and b(j) the j-th coefficient of an n - vector B, 



j— 1, . . . , m . 
If, on the 
define the 
formulated 


Only the data 
other hand, two- 
constraints , 
in the way 


A and B are 
■sided bounds 
then linear 


requested by EMP. 
are to be used to 
constraints are 




x 9 *a ( 1 ) 


= 


b ( 1 ) , 






b(me+l ) 


x’*a(me) 

<= x ’ *a ( me+ 1 ) 


< = 


b(me) , 

b( m+1 ) 


i 




b(m) 


<= x* *a(m) 


< = 


b(m+m-me) 





General constraints, i.e. at least one of the constraints 
is nonlinear. It is possible in practice, that all 
restrictions possess the same structure and depend at 
most on an index * i* for the i-th constraint. In this 
case only some FORTRAN statements for defining G(i,x) 
must be provided by a user. 
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The mathematical programming algorithms require the 
evaluation of gradients of all problem functions. If they 
cannot be formalated automatically by EMP, i.e. if the 
problem functions are neither linear nor quadratic, then the 
user has two options. In the first case, he requires the 
numerical evaluation by scaled forward differences* and no 
additional input is necessary. In the second case, he prefers 
to evaluate them analytically by a sequence of suitable 
FORTRAN statements. In general it can be recommended to use 
the numerical approximation option. Analytical derivatives 
should be determined only if difference quotients produce 
significant round-off errors or if the additionally required 
n function evaluations are too expensive compared with one 
gradient evaluation. 

An additional option of EMP is the possibility to perform 
a statistical analysis, if the objective function describes a 
parameter estimation problem subject to the L2-norm. In this 
case the following quantities are evaluated and displayed on 
request : 

Variance/covariance matrix of the problem data 
Correlation matrix of the problem data 

Standard deviation for the previously calculated parame- 
ters 

Estimated variance of the residuals 

Confidence intervals for the individual parameters sub- 
ject to the significance levels 1,5, and 10 per cent 

Note that it is always allowed to direct the output of the 
optimization routines and the mentioned statistical analysis 
to a user provided file. 
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3. Algorithms for solving mathematical programming problems 



Depending on the type of the problem functions, EMP 
creates either a general nonlinear programming, a nonlinear 
data fitting, a multicriteria optimization, a quadratic pro- 
gramming or a linear programming problem. In all cases, the 
chosen algorithm is capable to take into account two-sided 
bounds for the constraints and lower and upper bounds on the 
variables after a suitable reformulation of the model, if 
necessary. The linear and quadratic programming problems are 
formulated only when all constraints are linear. 

The following FORTRAN algorithms are available to solve 
any of the mentioned mathematical programming problems. 
Detailed user documentations are contained in EMP and can be 
displayed interactively. 



a) NLPQL (Schittkowski (1985/86)): 

NLPQL realizes a sequential quadratic programming ( SQP- ) 
method for solving the general nonlinear programming problem 
with continuously differentiable objective and constraint 
functions. The algorithm formulates a sequence of quadratic 
programming problems by linearizing the constraints and 
approximating the Lagrange function quadratically . By solving 
these subproblems, a search direction for the variables and 
the multipliers is obtained. Subsequently a line search is 
performed along that direction, to get a new iterate, i.e. a 
new guess for the variables and multipliers. The merit 
function to calculate the steplength parameter, is an 
augmented Lagrangian or an Ll-exact penalty function alter- 
natively. Then a new approximation matrix for the Hessian of 
the Lagrange function is updated by the BFGS-quasi-Newton 
f ormula . 

The SQP-method was first proposed by Wilson (1963) and in- 
vestigated by Han (1976,1977) and Powell ( 1978a , 1978b ) in 
more detail. In particular they proved that the algorithm 
converges superlinearly when starting sufficiently close to 
an optimal solution. The mathematical algorithm of the used 
implementation is described completely in Schittkowski (1983) 
together with a global convergence result. This theorem 
states that starting from an arbitrary initial point, a Kuhn- 
Tucker point of the problem will be approximated, that is a 
point satisfying the necessary optimality conditions. 

For the problem type under consideration, i.e. for smooth, 
small scale and eventually highly nonlinearly constrained 
problems, the SQP-method can be considered as the most 
powerful solution technique, see Schittkowski (1980) or Hock, 
Schittkowski (1983) for an extensive general purpose 
comparison with other available methods, or Schittkowski 
(1984/86) for performance results of NLPQL. Despite of its 
excellent average efficiency and reliability, an SQP- 
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algorithm can fail under certain circumstances, e.g. if 

the problem functions are not continuously differentiable 
or even not continuous in the feasible region defined by 
the lower and upper bounds on the variables, 
the problem functions can not be evaluated within a 
sufficiently small accuracy, 

the problem is badly scaled, ill-conditioned or does not 
satisfy the constraint qualification, i.e. the linear 
independency assumption of gradients of active 
constraints , 

the starting point is too far away from a solution, so 
that the solution progress is too small, 

the problem possesses several local solutions and the 
approximated one is not appropriate. 

Typical reaction of an SQP-method in one of the above 
situations, is a slow convergence indicated by attaining the 
maximum number of iterations, or the inability to find a 
suitable steplength. In such a case, EMP allows to choose 
other solution methods to overcome the mentioned difficul- 
ties . 

NLPQL is now availble in the IMSL-library under the names 
NCONF and NCONG . 



b) E04VDF (NAG: Gill e.al. (1983a)): 

The underlying mathematical idea is very similar to that 
in NLPQL, i.e. E04VDF realizes a sequential quadratic pro- 
gramming method. In contrast to NLPQL, the Hessian of an aug- 
mented Lagrangian is approximated instead of the Lagrangian 
itself, and a ’hot* start is possible for the quadratic pro- 
gramming subproblem, to get an initial guess for the first 
basis. It is allowed to define linear constraints separately 
and all constraints possess upper and lower bounds. 

EMP generates the same FORTAN subroutines NLFUNC and 
NLGRAD, which are used to solve the problem by NLPQL. 
Safeguards prevent unnecessary evaluations of function 
values. If two-sided bounds for constraints are not defined 
by the user initially, reasonably large values are inserted. 



c) ELL ( Schittkowski (1985)): 

The algorithm ELL belongs to a class of methods, that do 
not depend on 

an initial guess for the variables, 
any multiplier approximations, 
scaling properties of the problem functions, 
differentiability assumptions. 

In this case, it will be tried to solve the nonlinear 
programming problem by a variant of the ellipsoid-method, 
which was proposed by Shor (1977) for solving convex 
mathematical programming problems. Starting with an ellipsoid 
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containing the rectangle defined by lower and upper bounds on 
the variables, a sequence of ellipsoids is constructed with 
volumes decreasing linearly to zero. Any new ellipsoid is 
uniquely determined by the requirement, that it should 
contain one half of the previous one with minimal volume. The 
cutting half-space is determined by the gradient of a special 
penalty function. Iterates are the centers of the ellipsoids. 

Originally, an ellipsoid method is only capable to solve 
unconstrained problems. Constraints and bounds for the 
variables are taken into account by an Ll-penalty function. 
The corresponding penalty factor must be defined by the user. 

The convergence speed of the ellipsoid method is at most 
linear and depends on the number of variables. Since 
multipliers are not approximated, a precise stopping 
condition cannot be given in the constrained case. However it 
is possible to predetermine the number of iterations to 
achieve a relative reduction of the volumes with respect to 
the starting ellipsoid. An additional termination parameter 
is the desired accuracy in constraint satisfaction. The 
default value is three digits and should not be increased too 
large . 

Theoretically, the method converges only if the underlying 
problem is convex. But the numerical results of Ecker and 
Kupferschmid (1983) show that the algorithm is quite robust 
also in the non-convex case. From the numerical point of 
view, the method is in general less efficient than an SQP- 
method, i.e. uses a larger number of function and gradient 
evaluations in particular when a high accuracy is required. 
But it can be used quite sucessfully to obtain a suitable 
starting point or to ’move away’ from an undesired local 
solution . 



d) M1FC1 (Lemarechal, Strodiot, Bihain (1981)): 

The algorithm solves unconstrained nonlinear programming 
problems either with a smooth or nonsmooth objective func- 
tion. Constraints are taken into account by formulating an 
Ll-penalty function, where the corresponding penalty para- 
meter must be determined by the user. If a gradient of the 
objective function or a constraint is to be calculated 
analytically at a non-dif f erentiable point, then at least a 
subgradient must be available. 

Proceeding now from an unconstrained optimization problem, 
the algorithm collects a certain user provided number of pre- 
viously computed gradients to formulate a bundle, i.e. an 
interior approximation of the eps-subdif f erential . A minimum- 
norm feasible point of the bundle is calculated then to get a 
search direction. Subsequently a line search is performed to 
obtain a new iterate. 

Besides of the penalty parameter and the number of 
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gradients determining the bundle, a user must specify several 
stopping tolerances and in particular a numerical figure that 
determines the expected decrease of the objective or penalty 
function, respectively. Numerical tests indicate that this 
number should be chosen very carefully, since the performance 
is sensitive to its choice. 



e) ZXMWD (IMSL: Fletcher (1972)): 

Problems with box constraints are transformed into 
unconstrained problems by using trigonometric functions. 
General nonlinear constraints are taken into account by an 
L2-penalty function. The unconstrained problem is then solved 
by a quasi-Newton method of Fletcher (1972). Smooth problem 
functions are required. 

Besides of the penalty parameter, the user is asked to 
specify a certain number of starting points which are 
generated randomly. Then about four iterations are performed 
by the quasi-Newton method and the five results with the 
lowest function values are then taken to continue the 
iteration. Thus the code can be used in particular when it is 
supposed that the problem possesses various local minima. 



f) MCO (Schittkowski (1986b)): 

The multicriteria problem mentioned in the previous sec- 
tion, is transformed into a nonlinear programming problem 
with one differentiable objective function, which is then 
solved by NLPQL. Four different attempts are available to 
obtain the transformed problem: (1) Sum of weighted 

functions 

(2) Sum of weighted functions with additional bounds on 
objective functions 

(3) Selecting one function, suppressing the other ones 

(4) Combination of ideal solution and the center of best 
value points 

The corresponding weights are defined interactively when 
executing the FORTRAN-code , and can be modified during the 
same session. A list of efficient points calculated during an 
interactive session, is kept in memory for further pro- 
cessing. Their number can be determined by the user among 
some other stopping tolerances used for NLPQL. 



g) DFNLP (Schittkowski (1985c)): 

Although many efficient special purpose algorithms were 
developed in the past to solve data fitting problems, only 
very few codes are available that are able to take into 
account nonlinear constraints. The program DFNLP solves 
constrained data fitting problems in LI, L2 and maximum norm 
and, in addition, min-max problems by transforming them into 
a suitable nonlinear programming problem. This problem is 
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then solved by algorithm NLPQL. 

(i) Ll-data fitting problem: We introduce 1 additional 

variables z ( 1 )»•••> z ( 1 ) to transform the original 
problem into a nonlinear programming problem with 
objective function 

F(x,z) = z ( 1 ) + ... + z ( 1 ) 

and 1+1 additional inequality constraints of the form 

-H j ( x ) + z(j) >= 0 , j=l,...,l , 

H j ( x ) + z ( j ) >= 0 , j = l, . . . ,1. 

Obviously both problems are equivalent. 

(ii) L2-data fitting or nonlinear least squares problem: By 

introducing 1 additional variables z(l), ... ,z(l), the 

least squares problem is transformed into a general 
nonlinear programming problem with objective function 

F ( x , z ) = z ( 1 ) * * 2 + . . . + z ( 1 ) * * 2 

and 1 additional equality constraints 

Hj (x) - z( j ) = 0 

j=l,...,l. It turns out that an SQP-method applied to 
the modified problem, retains some characteristic 
features of a special purpose least squares algorithm, 
in particular the usage of a Gauss-Newton direction 
combined with a quasi-Newton approach. More details and 
some computational test results are found in 
Schittkowski (1985). EMP will ask the user whether he 
has some impression on the size of the residual. If he 
expects a small residual problem, then it will be tried 
to reduce the influence of the quasi-Newton matrix as 
much as possible. Moreover he should consider the 
possibility to restart the solution algorithm by 

defining a low maximum number of iterations, and by 
solving the problem again with an initial point which 
is known from the previous run. The resulting numerical 
algorithm takes advantage of the special structure of 
the quadratic programming subproblems, so that 

especially problems with a large number of experiments 
do not lead to excessive calculation times. 

(iii) Maximum-norm data fitting: In this case only one 
additional variable z is required to transform the 
problem into a smooth nonlinear programming problem 
with objective function 

F ( x , z ) = z 



and 1+1 additional constraints 
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-H j ( x ) +. z >= 0 , j=l, . . . ,1, 
H j ( x ) + z >= 0 , j=l, . . . ,1. 



(iv) Min-max problem: Very similar to the above case, one 

additional variable z is introduced to obtain an 
objective function 

F ( x , z ) = z 

and 1 additional constraints of the form 



-Hj ( x ) + z >= 0 , j=l , . . . , 1 , 



to obtain an equivalent smooth nonlinear programming 
problem . 



h) DFELL ( Schittkowski (1986d)): 

DFELL realizes an ellipsoid algorithm for solving LI, L2 , 
maximum norm and min-max problems directly without 
transforming them to a constrained nonlinear programming 
problem as done in DFNLP. Constraints are taken into account 
by formulating an L2-penalty function. After some 
arrangements of data, the code ELL is executed to solve the 
problem . 



i) DFM1FC1 (Lemarechal, Strodiot, Bihain (1981)): 

Very similar to DFELL, the least squares problem is 
transformed into an unconstrained problem and then solved by 
M1FC1 . 



j) NLSNIP (Lindstroem (1983)): 

The algorithm solves nonlinear least squares problems 
subject to nonlinear equality and inequality constraints. The 
Gauss-Newton method is modified to handle constraints by an 
active set strategy. By solving the resulting linear least 
squares problem with linear equality constraints, a search 
direction is obtained. Using a penalty function subject to a 
norm that must be specified by the user, a line search is 
performed to get a new iterate. To stabilize the algorithm, a 
subspace minimization option is included in particular for 
large residual problems. Alternatively it is possible to 
provide second derivatives and the algorithm switches to 
Newton’s method automatically, if necessary, i.e. if the 
Gauss-Newton search direction does not lead to a sufficient 
success . 

First derivatives are either defined analytically or are 
computed numerically by forward differences. If a user 
prefers the usage of Newton’s method, second derivatives are 
approximated numerically. When starting NLSNIP, the desired 
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norm of the penalty function used for the line search, the 
usage of Newton’s method and some stopping tolerances must be 
specified . 



k) E04FDF (NAG: Gill, Murray (1978)): 

The algorithm solves unconstrained least squares problems 
by a modified Gauss-Newton method and uses only function 
evaluations. Gradients are approximated numerically. 

Constraints and bounds are not taken into account, and are 
simply omitted when solving a problem. For the evaluation of 
the problem functions, EMP generates the same subroutine 
called DFFUNC which is formulated also for the least squares 
algorithms DFNLP , DFELL and DFM1FC1 . This routine is then 
called by the special purpose subroutine required by E04FDF . 



1) E04GCF (NAG: Gill, Murray (1978)): 

The algorithm solves unconstrained least squares problems 
by a modified Gauss-Newton method and requires also gradient 
evaluations. A quasi-Newton method is used to approximate 
part of the Hessian of the objective function. 

Constraints and bounds are not taken into account, and are 
simply omitted when solving a problem. For the evaluation of 
the problem functions and gradients, EMP generates the same 
subroutine called DFFUNC and DFGRAD which are formulated also 
for the least squares algorithms DFNLP, DFELL and DFM1FC1. 
These routines are then called by the special purpose 
subroutines required by E04GCF. 



m) E04GEF (NAG: Gill, Murray (1978)): 

The algorithm solves unconstrained least squares problems 
by a modified Gauss-Newton method and requires also gradient 
evaluations. A difference formula is used to approximate part 
of the Hessian of the objective function. 

Constraints and bounds are not taken into account, and are 
simply omitted when solving a problem. For the evaluation of 
the problem functions and gradients, EMP generates the same 
subroutine called DFFUNC and DFGRAD which are formulated also 
for the least squares algorithms DFNLP, DFELL and DFM1FC1. 
These routines are then called by the special purpose 
subroutines required by E04GEF. 



n) ZXSSQ (IMSL: Brown, K.G., Dennis, J.E. (1972)): 

The unconstrained least squares problem is solved by a 
modification of the Levenberg-Marquardt algorithm. Gradients 
are evaluated numerically. Some options are available to 
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modify the Marquardt parameter before starting the algorithm. 
Constraints are not taken into account. 



o) QL ( Schittkowski (1986e), Powell (1983)): 

An efficient quadratic programming algorithm is required 
not only for solving special problems of this type, but also 
for implementing any SQP-method. In the present case, an 
algorithm developed by Powell (1983) was modified to satisfy 
additional requirements, e.g. to handle upper and lower 
bounds separately or to exploit initially given Cholesky- 
factorizations . The mathematical method was proposed by 
Goldfarb and Idnani (1983) and can be characterized as a dual 
approach. Starting from a Cholesky-decomposition of the 
objective function matrix, the minimum of the unconstrained 
problem is found immediately. Successively all violated 
constraints are incorporated into a 'working set' defining a 
certain subspace, where the minimum of the objective function 
is recalculated. It might be necessary to drop constrains 
from the working set, until the optimal solution is reached. 



p) E04NAF (NAG: Gill e.al. (1983b)): 

The program solves quadratic programming problems by the 
primal method of Gill e.al. (1983b), which can be charac- 
terized as an active set strategy. The objective function 
matrix must be symmetric, but is allowed to be indefinite. 
The algorithm allows a 'warm' start and possesses many 
additional features, so that the reader is referred either to 
the original report or the program documentation included in 
EMP, to get more information. 



q) LP (Schittkowski (1986f)): 

EMP is not designed to solve large and sparse linear 
programming problems or linear programming problems with a 
special structure. A specific modelling language as known 
from linear programming applications, is therefore not 
available. The input of sparse matrices is facilitated in the 
way that the user is allowed to to insert only the non-zero 
entries of matrices and vectors, or to define the matrix 
values by one arithmetic expression. In the latter case, the 
elements may be calculated somewhere else, so that only 
values of a two-dimensional array can be transferred. The 
linear program is solved by the simplex method. 



r) E04MBF (NAG: Gill e.al. (1983b)): 

This code is based on the quadratic programming method 
E04NAF and uses the primal method included there. Linear 
constraints must be defined with two-sided bounds. 
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s ) ZX3LP ( IMSL ) 

The code realizes a simplex algorithm. A transformation of 
the given problem is required, since the code accepts only 
lower zero bounds for the variables. 



Linear least squares problems are transformed into quadra- 
tic programming problems. Since the resulting positive semi- 
definite matrix of the objective function is calculated in 
double precision arithmetic, the additonally introduced 
round-off errors might be tolerable. 
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4 • Commands of EMP 



All actions of EMP are controlled by commands. They can be 
displayed in form of a menue by typing the questionmark ’ 
From the viewpoint of a user, they should be considered as 
self-contained programs, i.e. they can be used independently 
from each other. If problem data are generated or altered, 
the corresponding data will be saved on a file provided by 
the user. Whenever EMP executes an operating system command, 
e.g. for compiling and linking a FORTRAN program, a separate 
subprocess is generated and executed. Subsequently EMP is 
continued, but the user will not realize such a situation. 

Once a command is specified, EMP asks the user to answer 
further questions e.g. in the form 

* Analytical derivatives (Y/N): 

In the above example, a user has to type either *Y f for yes 
or * N* for no. It is always assumed that the less probable 
answer is 9 Y 9 , so that in this case a has to be typed 

exactly and any other input is interpreted as * N*. But 
whenever a user does not know the precise meaning of the 
brief information given, he is allowed to press , ? > . He 
obtains then more detailed information before he has to 
answer the question again. In the same situation it might 
happen that a user wants to cancel the present execution of a 
command, e.g. if he notices a severe previous input error. By 
typing the sign * <* he will return immediately to main menue. 

By the following subtopics, only a rough description of 
available EMP commands will be given to understand their 
basic operations. Their particular usage is self-explained by 
sub-menues, additional information displayed on the terminal 
whenever it seems to be necessary and the help-option of EMP. 



a) INFORMATION: 

Displays some information on the screen to understand the 
system, the mathematical models and algorithms, or the 
available commands. The displayed text is identical with this 
documentation which is used to extract the desired sections. 
Alternatively the whole documentation can be formatted and 
printed on request by a special command option. 



b) GENERATE: 

In an interactive way, the user will be informed how to 
construct a new mathematical optimization problem. After 
input of the problem name and, optionally, some additional 
information on the practical background, the type of the 
problem and its functions must be specified. Then some 
dimensioning parameters are required, e.g. number of 




146 



variables and constraints. User provided declaration 
statements, constant data, scaling parameters or parameter 
estimation data follow whenever required for constructing the 
actual problem under consideration. 

Besides the input of the function declaration statements, 
it is possible to define additional FORTRAN statements which 
will be inserted either into the main program or the 
subroutines evaluating the problem functions and gradients. 
They are placed directly after assigning actual iteration 
values to the variables, which can therefore be modified or 
used to prepare some initial data before calling the 
optimization algorithm or calculating the function and 
gradient values, respectively. Further statements can be 
inserted into the main program after executing the 

optimization subroutine, e.g. to process the achieved 

results. In this case, the user provided variable and 
function names contain the computed numerical values. 
Moreover the user may add some arbitrary FORTRAN subroutines 
or functions, which will be appended to the generated source 
code . 

The input of additional FORTRAN statements is to be 
illustrated in form of a sketch. The blocks surrounded by * 
are optional and can be inserted by the user in form of 
arbitrarily many FORTRAN statements. 



Main Program: 

Declaration statements 

******************************** *************************** 
* * 

* DECLARATION AND FIRST EXECUTABLE STATEMENTS * 

* * 
*********************************************************** 

Preparation of initial data, addressing starting values 
to variables 

*********************************************************** 
* * 

* ADDITIONAL EXECUTABLE STATEMENTS * 

* * 
*********************************************************** 

Call of the mathematical programming algorithm 

t********************************************************** 
* * 

* PROCESSING THE ACHIEVED RESULTS * 

* * 
*********************************************************** 



Storing results on a file 
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Subroutine evaluating function values: 



Declaration statements 



DECLARATION AND FIRST EXECUTABLE STATEMENTS 



Addressing iteration values to user-provided variable 



* 

ADDITIONAL EXECUTABLE STATEMENTS * 

* 



Evaluation of problem functions 



Subroutine evaluating gradient values: 



Declaration statements 



* DECLARATION AND FIRST EXECUTABLE STATEMENTS 

* 



Addressing iteration values to user-provided variable 



* ADDITIONAL EXECUTABLE STATEMENTS * 

* * 

Evaluation of gradients 



********u************************************************** 
* * 

* ADDITIONAL SUBROUTINES OR FUNCTIONS * 

* * 
tt*t*t*t******t***t*tt***t%*t*tt*****t*t**************ttt*% 



In the above sketch, it is assumed that the used 
optimization program requires the evaluation of functions and 
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gradients in two different subroutines. It is, however, 
possible that functions and gradients are to be calculated in 
one subroutine or even in two subroutines separately for 
objective function(s) and constraints. In this case the 
additionally defined FORTRAN statements are all inserted into 
each subroutine that evaluates functions and gradients 
together . 

Except for the quadratic or linear programming problems, 
it is necessary to provide initial estimates for the 
variables and to declare upper and lower bounds. If all 
initial values and bounds are identical or depend at most on 
an integer * i ’ , then the input of these data can be reduced 
to only one line by typing a suitable arithmetic expression 
depending on an ’ i ’ . 

The evaluation of the problem functions depends on their 
specific type defined earlier. If a function is linear or 
quadratic for example, then only the corresponding matrices 
or vectors must be defined. In the general case, the input of 
those functions is facilitated, which can be evaluated within 
one line. Otherwise an arbitrary number of FORTRAN statements 
must be typed by the user for addressing a numerical value to 
a function, i.e. for assigning a numerical value to a 
function name. 

When defining a problem function by a couple of FORTRAN 
statements, a user should avoid conflict with existing labels 
and names. In the first case, the most simple way to prevent 
an overlapping of labels, is the restriction to numbers 
between 1 and 1000. The first character of any variable 
should satisfy the FORTRAN convention, i.e. should be in the 
set ’a* - ’h’ , ’o’ - ’z’. The numerical algorithms require 
additional real variables for some internal organisational 
reasons. But all real variables defined by the system, 
consist of six characters, so that the easiest way to avoid 
conflicts, is the usage of real names with less than six 
signs. Integer values like I, J, K are used by EMP only 
within ’closed loops’, i.e. for control constructs where a 
user cannot insert his own statements. Therefore they are 
free for defining function statements. As above, all names of 
FORTRAN-subroutines reserved by EMP, consist also of six 
characters and any conflict is avoided by using at most five 
characters for user provided subroutine or function names. In 
a similar manner, named COMMON-blocks required by an 
optimization algorithm, are identified by six signs. 

The last action of GENERATE is the request to determine 
those file names, which contain additional subroutines and 
which are to be linked with the FORTRAN code generated by 
EMP, and the corresponding solution algorithms. Note however, 
that GENERATE does not write the mentioned code, but saves 
all collected data on a user provided file for further 
processing . 
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c) SOLVE: 

This commands starts the numerical solution of a problem 
that was previously constructed by GENERATE. After input of 
the problem name, a user has the option to require the 
display of all available codes that could solve his 
optimization problem. The list contains a certainty factor 
for each proposed program which indicates a very rough 
approximation of a measure for the numerical performance of 
an algorithm. A value of 100 is the maximum attainable degree 
of belief, whereas a value of 0 indicates a very poor 
performance of the algorithm on previous runs. The numerical 
values of the certainty factors are defined and updated in 
three different ways: 

a) Initially every code obtains a certain permanent default 
value of the system author which is based on his own 
subjective experience. 

b) Whenever a code could solve a problem succcessf ully , so 

that the stopping criteria are satisfied subject to the 
provided tolerances, the corresponding permanent 

certainty factor of the code is increased. If, on the 
other hand, a failure is reported, then the permanent 
certainty factor is decreased. The factor is not altered 
at all if the iteration was interrupted due to the fact 
that the maximum number of iterations was attained. 

c) When starting the solution of a problem, a local 
certainty factor is defined which gets the value of the 
permanent one, and all previous numerical solution 
attempts are investigated. If the problem could not be 
solved by a specific algorithm, then the local certainty 
factor is decreased significantly, and enlarged 
otherwise . 

The local certainty factors are displayed to support the 
selection of a suitable code. It is hoped that the choice of 
these factors reflects the influence of special model 
structures and that EMP is capable to learn, i.e. to propose 
better and better solution methods in dependance on the 
problem to be solved. It is selfevident that a user may 
reject the algorithm which got the largest certainty value, 
and to choose another one. 

Subsequently some additional decisions must be made by a 
user, e.g. the choice of a suitable output flag. It is 
possible that a previously computed and eventually inaccurate 
approximation of the solution is available. Then the user is 
asked whether he wants to accept this guess as a starting 
point for the actual run or not. If some additional output 
from the underlying mathematical programming algorithm is 
required, then the information is displayed on the terminal 
in original form, in particular without individual user- 
provided names for functions or variables. 
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The generated FORTRAN code is written on a text file with 
the name * EMPCODE . FOR * . The code consists of a main program 
and, if the problem functions are nonlinear, two subroutines 
for evaluating problem functions and corresponding gradients 
in a form required by the selected optimization code. EMP 
compiles this program, links it with a chosen nonlinear 
programming algorithm and eventually some user files, and 
executes the resulting program. All numerical results, 
performance data and termination reasons are stored 
automatically in the underlying data base and are available 
for further processing. Afterwards the main-menue of EMP is 
displayed again and the user could choose additional actions, 
e.g. to investigate the obtained results. Note that after 
leaving the system, the last generated FORTRAN program is 
still available and could also be used furtheron indepen- 
dently from EMP. It is possible to direct the output of an 
optimization program to an existing file on request. 



d) EDIT: 

Command EDIT allows to modify all available problem data 
and function statements interactively. After defining the 
desired problem name, the user has to specify whether he 
prefers the usage of the SUSY editor or the operating system 
editor. Then a menue allows him to indicate the specific data 
or functions that are to be edited. 

If the SUSY editor is chosen, then a user must know the 
commands that are allowed to edit lines and which consist of 
certain signs to be typed at the first position of the actual 
input line. The following options are available: 

A Add a line (following directly the displayed one). 

M More than one line are to be added. 

R Replace the displayed line completely. 

S Scratch displayed line from the file. 

E Exchange characters of the displayed line by non- 
blank characters typed in the input line. 

I Insert additional characters at positions after the 

last blank sign of the input line. 

B Blanks are to be inserted proceding the position 

indicated by an arbitrary non-blank character in the 
input line. 

D Delete characters marked be arbitrary non-blank signs 

in the input line. 

N Next line. 

Ni Proceed to the i-th line following the displayed one. 

P Previous line. 

Pi Proceed to the i-th previous line. 

H Halt editing the actual block of statements. 

< Return to the beginning of the EDIT-command . 
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e) TRANSFER: 

It is sometimes desirable to generate or modify a problem 
that is very similar to an existing one, e.g. if a new 
problem is to be solved with the same objective function, but 
some other constraints or constant data. A transfer procedure 
allows to transport information from a source problem to a 
destination problem. If the latter one exists in the problem 
base, then the corresponding data will be overwritten. 
Otherwise a new problem is generated with the same problem 
parameters and the data to be transferred, are inserted. 

A user should apply the TRANSFER-command very carefully 
when the underlying problem structure is to be altered. 
Particular attention is necessary if the number of variables 
or functions is different in both problems. 



f) DELETE: 

Any problem that exists in the data base, can be deleted 
on request by the user. Unless EMP is not stopped by the 
HALT -command , a deleted problem is still available on a 
backup file with name * EMPBACKUP . DAT * , and can be retrieved 
in an error case. In this case the system should be left by 
, <ctrl>-Y > and the backup file is copied to the data base 
file defined initially when starting EMP. If two problems 
possess the same name, only the first one will be deleted. 



g) REPORT: 



This command displays problem data and function statements 
on the one hand and numerical solution data on the other. In 
both cases it is possible to get a complete list of all 
available problem names together with some condensed 
information, i.e. problem parameters or performance results. 
After typing the problem name, a user is asked to specify the 
data he wants to see. In principal he will be informed on 
every parameter, data or statement, that was used to generate 
the problem. 



If the problem was solved numerically, it is possible to 
get a detailed report on all obtained results. Moreover a 



history can be displayed on request, 
data are shown, e.g. the number of 
evaluations, that were used to obtain 
If a problem could not be solved 
user is not satisfied by the obtained 
results, then a failure analysis can be required that shows 
the user not only possible reasons for failures or 
unsatisfactory results, but also some remedies to overcome 
this situation, see the description of the command FAILURE 
ANALYSIS. 



complete optimization 
Then some performance 
function and gradient 
the current solution, 
successfully or if a 



An additional feature of REPORT is the possibility to ask 
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for a line printer output of all problem data and all 
numerical results for any problem in the data base. The 
corresponding data are written on an auxiliary file with name 
* EMPDRUCK . DAT * that is not deleted after leaving EMP and that 
can therefore be used also for other purposes. 



h) ASSORT: 

Whenever a new problem is generated, it will be placed at 
the first position of the corresponding problem file or data 
base, respectively. However it might be desirable to sort the 
data base to get a more convenient problem listing on the 
screen when displayed by the REPORT-command . Therefore 
problems can be sorted with respect to a few possible keys, 
e.g. problem name, dimension, or problem type. These keys are 
listed on the screen on request and are initiated by typing a 
specific character. 



i) FAILURE ANALYSIS: 

It is quite usual, that a problem could not be solved 
successfully so that the stopping criteria are satisfied 
within some given tolerances. All algorithms incorporated in 
EMP, report error situations by certain flags that are 
displayed in the optimization history table among other 
performance data. A value of *0* indicates a successful 
return as defined above, a value of 'l* that the maximum 
number of iterations was attained, and a value greater than 
* 1* a specific error situation depending on the algorithm 
chosen. To find out the meaning of the displayed numerical 
figure, the command FAILURE ANALYSIS can be executed. In 
general, the corresponding information is taken from the user 
guides of the corresponding programs. 

In some cases, e.g. if the code NLPQL or any other one 
based on NLPQL is chosen, some heuristic knowledge is 
available on possible origins of the failure. This knowledge 
is represented in form of rules with certainty factors and by 
executing the reasoning process, some possible reasons for 
the failure can be displayed in the order of their 

significance. Moreover these reasons are exploited again by 
rules to propose some alternative remedies. The user could 
investigate these recommendations and follow one of them 
whenever it seems to be profitable. An additional option is 
the possibility to require some explanations for 
understanding the reasoning process. 

A typical rule might be of the form 
IF MUL> 1 . E + 5 
THEN RSNCQU WITH 30 
reading as 

* If the multiplier values are very large, 
then there is some evidence that the constraint 
qualification might be violated . 9 
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The above rule asserts that a certain consequent is 
emphasized with a given degree of uncertainty, which could 
also be supported by other rules. Moreover there are other 
rules to conclude a possible remedy, e.g. 

IF RSNCQU WITH 50 TO 100 

THEN REMNEWIN WITH 95 

reading as 

9 If the violated constraint qualification might be 
a possible reason for false termination, 
then it is recommended to use a new starting value 
with a high certainty . 9 

Consequents must consist of so-called actions, i.e. an 

arbitrary and eventually empty sequence of SUSY-commands , 
where antecedents are either logical expressions or again 

actions which have to be verified up to a certain range of 
uncertainty in order to be taken into account. 



,i) OPERATING SYSTEM COMMAND: 

When starting EMP, the file containing the physical data 
base must be read and the corresponding data must be 
processed in a specific way. A few other data management 
procedures are invoked by the interpreter, so that some 
execution time must be spent for each call of EMP depending 
on the size of the data base. To avoid the waiting time if 
only a few operating system commands are to be executed, a 
user may require the direct input of a sequence of operating 
system commands. By an empty input, the commands are sent to 
the operating system and executed immediately without leaving 
EMP. After finishing the last command, the main menue is 
displayed again. 



k) USER PROVIDED SUSY COMMANDS: 

Although it is tried to organize EMP in a way that all 
possible actions can be performed by activating suitable 
menues , there might be the possibility that a user wants to 
operate directly on the underlying data base. By choosing 
this command, he will first be asked whether he wants to be 
informed on a part of the SUSY documentation and whether he 
wants to see the data structures defined in the main program 
of EMP. It is an important assumption to know the names of 
available variables, in particular of those used for con- 
structing the general data structure of an optimization 
problem with the name 'prob*. Subsequently any valid SUSY 
command may be typed which is executed immediately. By an 
empty input, the interactive definition of SUSY commands is 
stopped and the main menue is shown again. 

Possible reasons for the interactive input of SUSY- 
commands are sorting of problems by other keys, searching for 
special strings, or, in particular, the copy of optimization 
problems from one database to another. In this case, the 
commands 
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REWRITE AUX 

WRITE PROB.. TO AUX 

write the actual and all following problems of the data base 
to the auxiliary file AUX. After stopping the system and 
calling it again now with another physical data base, i.e. 
another initial file name, the interactive commands 

RESET AUX 

READ PROB . . FROM AUX 

read all problems from AUX and insert them into the new data 
base directly after the actual problem. 



1) HALT: 

By pressing ’H* for activating the command HALT, the 
system will be left and the user returns to his operating 
system. The actual data base will be stored on the file 
provided initially by the user. A backup file with the name 
* EMPBACKUP . DAT * is also available containing the last updated 
problem base. 
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EFFECTIVE ALGORITM FOR GLOBAL OPTIMIZATION 
WITH PARALLEL COMPUTATIONS 

Roman G. Strongin, Yaroslav D. Sergeev 
Gorky State University 
Gorky, USSR 



1 . INTRODUCTION 

Appearance of multiprocessor computer systems and local 
computer networks gives wide latitude for constructing opti- 
mization techniques using parallel iterations including 
simultaneous (due to many processors) computations ( trials ) 
of function to be optimized values at several points in a 
parameter space. Each trial appearing in sach parallel ite- 
ration could be performed at a separate processing unit using 
the same (shared or copied) program. 

Let time-consumption of a single trial be substantially 
greater than time nedeed to select (following a decision rule) 
a group of points for next parallel iteration. Then it is 
also possible to neglect transfer times for exchanges between 
tigtly coupled processors (transfers of points of trials and 
reception of function values computed in these points). In 
this case the effectiveness of the parallel algorithm could 
be estimated by total amount of trials nedeed for solving a 
problem and by time period nedeed for this solution process 
(i.e. estimation by two criteria). 

For a net formed by points of trials in a parameter space 
generated by a parallel algorithm it is naturally desirable 
to possess the same (or nearly the same) density as in the case 
of effective sequential algorithms (i.e. introduction of pa- 
rallelism should not cause an appearance of redundant trials). 
With account of this condition it is desirable to use the 
largest possible number of processors to speed the searching 
for function optimum. 

Following the above ideas we suggest some new parallel 
algorithms for global optimization. These methods are genera- 
lizations of effective sequential schemes from Strongin (1978) 
and Strongin and Markin (1986). 

2. PARALLEL ALGORITHM FOR ONE-DIMENSIONAL UNCONSTRAINED 
PROBLEMS 



The suggested algorithm for global minimization of a 
function f (x) , x £ /a,b / , based on the assumption that at 
any 1+1 - st iteration (step) could simaltaneously be com- 
puted p = p(l) function values at the points 
k+1 k+p 

A ) ... ) A , 

where k = k(l) is a total number of function values computed 
in the course of 1 previous steps, i.e. 
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k(l) = p(1) + ... + p(l). 

Trials of the first step are carried out at any arbitrary 

p = p(1) ^ 1 internal points x^ , . . . , x p of the given in- 
terval (a,b). The choice of points for any subsequent ite- 
ration is determined by the following rules: 

"1 k 

(a) points x , ••• , x from previous trials are re- 
numbered using subscripts in the following way: 

a = x Q < Xx| . . . < x ± < . . . < x k < x k+1 = b (1 ) 

and associated with values z^ = f (x^) , 1 i ^ k , computed 

at these points. The ordered sequence (1) is complemented by 
auxiliary points Xq = a and x k+ x] = b (values Zq and 

z k+/ ^ are undefined) ; 

(b) the following value is calculated: 

M = max •£ /z^ - z i _^ 1 /(x i - 1 < i *k}. (2) 

If (2) is undefined or equal to zero, it is assumed that M = 1 ; 

(c) for each interval (x^ _ ^ , x^) , 1 ^ i £ k+1 , the 



value R(i) (called characteristic ) is computed, where 

R(i) = (x. - x i _ 1 ) + (z ± - z i _ 1 )W(x i - x ±-1 ) - 

- 2(z ± + z ± _ 1 )/rM , 1-c i i k , (3) 

R(1) = 2(x 1 - x Q ) - 4z^/rM , (4) 

R(k+1) = 2(x fc+1 - x k ) - 4z k /rM (5) 

(here r is a parameter, with a value greater than 1); 

(d) all characteristics are sequenced in the order of 
decrease : 

R(i 1 ) * R(i 2 ) ^ ... ^ R(ij ) * ... * R(i k+1 ) ; (6) 



(e) next p trials of the 1+1 -st step are carried out 

at the points x k+ ^ , ••• , x k+ ^ , where x k+ ^* = S(i.) and 

0 

i. , 1 - d - p 9 are indices from the sequence (6), 

J 

S(i) = (x ± + x i _ 1 )/2 - sign(z^ - z ± _^ )(/z ± - 

- z^/VT^^/Zv , ( 7 ) 

1 i - k . If i = 1 or i = k+1 the second term in the 
above formula is omitted. 

It is supposed that 

P = P(l) - k(l) + 1 (8) 

for any value of 1^1. The stopping rule abrupts the 
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f k \ 

sequence |x j if the condition x^ - x^ ^ * B is met, where 
t = i^ from (7) and B ^ 0 is a given tolerance. 

3. CONDITIONS FOR NON REDUNDANT SEARCH 



It should be noted that the decision rules of the above 
algorithm are exactly the rules of the sequential method from 
Strongin (1978), if p(l) = 1 , 1^1. Next theorem states 

some features of comparative efficiency of these two schemes 
(parallel and sequential) for the case, when in (2) and (7) 
n = 1 (this value have to be used if the function to be mini- 
mized is Lipschitz function). 

THEOREM. 

Let and •( y m } be sequences of trial points gene- 

rated correspondingly by parallel algorithm with coupled trials 
(i.e. p(1) = 1 and p(l) = 2 , 1^1) and sequential algo- 

rithm while minimizing Lipschitz function with constant K 
(and tolerance in the stopping rule B = 0 - infinite search). 

Then : 

(1) {x k } = { y m } if r ^ 2 and constant M ^ K is 

plugged into the formulas (3)-(5 ),(7) of the decision rule; 

(2) |x k }fl W = { y m }f\ W if r ^ 2 and f(x) = K/x - x*/, 

x fc WC (a,b) , where x H € W is a point of absolute minimum 
of f(x) in /a, b./. 

Thus information about the results of previous iterations 
allows to determine not less then two points of next trials 
with the same efficiency (no redundancy) as in the choice of 
single trial in purely sequential algorithm. 

4. GENERALIZATION FOR MULTIDIMENSIONAL CASE 



The nested optimization scheme: 



min 

a^s? x 1 ^b 1 



min f (xx. , ... , •^••jvr) 

o & -y- K ^ 

a N~ X N “ N 



( 9 ) 



allows to apply the suggested parallel algorithm for multidi- 
mensional optimization in a hyperinterval (by one dimensional 
optimization of nested one dimensional subproblems). 

As follows from the theorem the above algorithm with N 
coupled trials along with nested scheme (9) allows to use 2^ 
processors without causing redundancy (here N is a dimensio- 
nality of an optimization problem). In this case every itera- 
tion for any external problem (i.e. minimization in /au ,b^/ ) 
originates two nested ( internal ) problems. Each of these nested 
problems could also be optimized (in /kp,b p/ ) by parallel 
algorithm and so on. ^ ^ 

To illustrate the possibilities of the proposed technique 
Table 1 presents some results of computer simulations of the 
search processes for two two-dimensional (i.e. N = 2 ) test 
functions with various number of processors (from 1 up to 9). 

In these experiments parameter r = 1.7 and relative tolerance 
in the stopping rule B = 0.01 . Analizing the Table note that 
condition (8) causing equality p(1) = 1 could force some pro- 
cessors standing idle in partiqular iterations. 
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TABLE 1 Results of computer simulation 



Number 
of pro- 
cessors 


Parallelism 
Ex- In- 
ter- ter- 
nal nal 


Number 
of steps 
(exter- 
nal) 


Total ' 

number 
of 

trials 


lime 


Speeding 


Redun- 

dancy 


1 


1 


1 


1-st 

21 


test 


function 

386 


386 


1 


1 


2 


1 


2 


21 




449 


235 


1.64 


1.16 


2 


2 


1 


22 




407 


221 


1.75 


1.05 


4 


2 


2 


21 




459 


127 


3.04 


1.19 


3 


1 


3 


21 




4-92 


185 


2.09 


1.27 


3 


3 


1 


25 




44-9 


176 


2.19 


1.16 


6 


2 


3 


23 




540 


106 


3.64 


1.4 


6 


3 


2 


22 




460 


93 


4.15 


1.19 


9 


3 


3 


15 




357 


54 


7.15 


0.92 


1 


1 


1 


2-nd. 

25 


test 


function 

526 


526 


1 


1 


2 


1 


2 


25 




541 


283 


1.86 


1.03 


2 


2 


1 


17 




373 


210 


2.5 


0.71 


4 


2 


2 


17 




379 


108 


4.87 


0.72 


3 


1 


3 


25 




573 


216 


2.44 


1.09 


3 


3 


1 


21 




474 


197 


2.67 


0.9 


6 


2 


3 


17 




411 


85 


6.19 


0.78 


6 


3 


2 


19 




441 


94 


5.6 


0.84 


9 


3 


3 


20 




498 


74 


7.11 


0.95 




FIGURE 1 Level sets for the 1-st test function 
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Test functions used are essentially multiextrimal. Level 
sets for one of these functions are presented in Figure 1. 

Another possible approach to multidimensional problems 
based on application of contineous space-filling curves y(x) 
mapping a unit interval /0,1/ on the x axis onto N -dimen 
sional hyper interval D • These mappings (called Peano curves 
provide the origin for the scheme (Strongin, 1978): 

min{ f (y) : y €■ D } = min { f (y(x)): x €■ /0,1/ j • 

Right-hand side of this expression could be computed by emp- 
loying the considered parallel algorithm with n = N . 

Similar parallel generalizations are possible for multi- 
dimensional multiextremal constrained problems by extending 
sequential algorithms from Strongin (1985) and Strongin and 
Markin (1986). 
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ABSTRACT: 

Scheduling of partially ordered unit time jobs on m machines, aiming at minimal schedule length, is 
known to be one of the notorious combinatorial optimization problems, for which the complexity 
status is still unresolved. Availabe results give polynomial time algorithms for special classes of 
partial orders and for the two machine case. The present paper includes a general representation 
theorem on the existence of schedules of a given schedule length and a corresponding algorithmic 
method that allows a unifying approach to some of the solved special cases. It includes also a 
general algorithmic scheme that may work in the 3-machine case and may eventually lead to a 
polynomial time algorithm (or at least to membership in CO-NP) for the general m-machine case. 
Finally, it gives some insight into so-called m-critical posets, which should play an essential role in 
a proof on the correctness of the intended polynomial time algorithm. 
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l. INTRODUCTION 

Scheduling of partially ordered jobs on m machines usually leads to hard (NP-complete) 
combinatorial optimization problems (cf. [17], [18] for an overview). This is true e.g. for 
minimizing the makespan (the schedule length) on two machines in the nonpreemptive case 
(even if only two types of job-durations are allowed [18]) and for minimizing the makespan on an 
arbitrary number of machines while all job durations are equal (unit time jobs) [18], [28]. Note 
that polynomial time solutions for the unit time case with integer starting points only correspond to 
polynomial solutions for the preemptive scheduling of jobs with arbitrary durations, as long as 
job durations are bounded by a polynomial in the number of jobs. This adds to the importance of 
the result that unit time jobs on two machines can indeed be scheduled in polynomial time, even if 
arbitrary release and due dates are involved [6], [9]. Polynomial time algorithms also exist for 
special 3-machine problems [9], they exist, furthermore, for any fixed number m of machines if the 
underlying poset is of bounded height [3] or of bounded width [1], [23], and it holds for arbitrary 

m, if the poset is a tree (forest) [14], an opposing forest [4] or an interval order [22]. However, 
inspite of intensive research by several scientists, the complexity status of the general m-machine 
unit time scheduling problem for m > 2, m fixed, still remains unresolved. In fact, it is one of the 
notorious problems in the field, mentioned already in the original list of ten basic open problems 
in complexity theory [10]. 

The present paper is a report on ongoing research concerning the general m machine problem. 
This research has been carried out by the authors now since about 5 years (cf. also [1], [23], [26]) 
and is guided by the conjecture that there should be a polynomial time algorithm for the general case 
with m being part of the exponent of the bounding polynomial. Within the paper, some normaliza- 
tions, theorems and algorithmic tools are presented and discussed in connection with illustrative 
examples, which might be of particular use for other scientists working on the subject. The 
organization of the paper is as follows: 

In Section 2, we start from a natural normalization of the problem (while covering some 
further generalizations such as profiles of available machines [3]) and then prove a quite general 
representation theorem (cf. [1]) that allows a decision on the existence of a feasible schedule 
of a given scheduling length by generalizing a well known bounding approach, known from the 
solution of the two machine case [8]. The representation leads to an algorithmic procedure for 
solving the general m-machine problem in 0(n m ) time depending, however, for m > 2, on the 
previous solution of a certain choice problem for which presently no general polynomial time 
solution is available. The choice problem is trivial for the 2-machine case and possible in an obvious 
way for some of the well-solved classes of posets such as interval orders and forests. The given 
algorithmic approach therefore makes it transparent, why the general case for m > 2 is so much 
harder to deal with then with the case m = 2 or certain special classes of partial orders. 
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The application of the presented algorithmic approach to some special cases of the problem 
under consideration is the subject of Section 3. We obtain results with the same performance 
bounds as the known solutions for the case m = 2 (even if release dates and due dates are involved), 
for special 3-machine problems and for the case of interval orders, where m is arbitrary. For forests 
and m arbitrary, we obtain a valid modification of the known solution, but while this has an almost 
linear performance bound, here the degree of the bounding polynomial is m. 

In Section 4, we try to get around the choice problem in the general form, as it occurs in 
connection with the representation theorem. What we hope for is the identification of a sufficient 
core of information w.r.t. the placing of jobs, deducible in polynomial time, such that only 
performing those choices needed in the representation theorem which can be settled on the basis of 
this information is sufficient in the following sense: there is a feasible schedule for a given problem 
exactly if the bounds, computed by using the settled choices only, do not lead to a contradiction. 
We then show that if this conjecture is true, there is a polynomial time algorithm for the m-machine 
problem. Also we use as an example a reasonable core of information that essentially is derived via 
a closure operator and involves upper and lower bounds on the starting and completion times of 
sets of jobs and can be obtained in polynomial time. We use this to study and solve quite involved 
examples in order to demonstrate the strength of the approach as well as some interesting 
phenomena in these examples. Actually, this closure operator seems to be sufficient for the 
3-machine case and should be of basic use also for the m-machine case in general. Possible 
extensions of the closure operator are also discussed. 

Finally, in Section 5, we concentrate on approaches to prove the intended results w.r.t. the 
bounding techniques, discussed in Section 4. In doing so, we show that the proof can essentially be 
restricted to so-called m-criticial posets [26], i.e. posets that can optimally be scheduled on m 
machines in t time units, but taking away whatever immediate ordering relation from the poset 
allows scheduling it in t - 1 time units. In a certain sense, such posets are the hardest cases to deal 
with. Also, fast identification of (maximal) subposets of this type would show the problem to be in 
CO-NP. Actually, there is some hope that such posets may be so special in there structure that a 
proof becomes much easier. In fact this idea turned out to be correct in the 2-machine case, for 
which the 2-critical posets were very special indeed [26]. Their characterization in particular yielded 
annother way to obtain known polynomial time algorithms for the 2-machine case and, furthermore, 
also made the much harder situation for m > 2 better apparent. In the present paper now, we give 
another proof for the characterization of the 2-critical posets, then add to the information given in 
[26] some more examples and insights concerning 3-critical posets and, finally, give some 
evidence, that the complete characterization of the 3-critical posets might be possible and also 
might settle the matter for the 3-machine case completely. 
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2. A BASIC REPRESENTATION THEOREM 

From a complexity theory point of view, the m-machine unit time scheduling problem is as follows: 



Given a partial order 0 = (A,0), where A is the set of jobs and 0 the ordering relation on A, 
and given some natural number t > 0, we ask whether there is a schedule for 0 completed by time t, 
i.e. whether there is a mapping T: A — > IN, with T(a) denoting the starting time of job a, a e A, 
fulfilling: 



(i) 0 < T(a) < t - 1 for all a g A, 

(ii) (a,P) g 0 and a * p implies T(a) < T(p) and 

(iii) | (a g A | T(a) = i} | < m 

for all 0 < i < t - 1. 

I.e. we ask, whether all jobs can be scheduled in such a way that ordering relations and 
machine availability are respected and all jobs are finished by time t. Of course, |A| < m*t is a 
necessary condition in order to be able to achieve this aim. In fact, one can easily normalize the 
problem to the case where |A| = m*t, which essentially leads to the question of whether it is 
possible to utilize all machines uninterruptedy up to time t . 



NORMALIZATION: 

The m-machine unit time scheduling problem [(A,0), t] is polynomially equivalent to the special 
case where the number of jobs is exactly the number of available time-machine slots, i.e. to the case 
|A| = nrt. The proof of the normalization is immediate by adding m*t - |A| independent (i.e. pair- 
wise incomparable) jobs to 0 = (A,0) while leaving t unchanged. These independent jobs can be 
filled in the free slots remaining open after a solution of the original problem. The transformation is 
of course polynomial (even linear) in the original input length. Note however that the reduction 
may not be valid for special classes of posets, cf. also the remarks given in Section 3. E.g. for 
trees, it would lead to forests in any non trivial case, while for forests, in turn, the transformation 
remains in the class. 

In the literature one if often concerned with more general problem formulations. For instance, 
there might be given release dates r(a) gN or due date d(a) < t, for which any schedule 
must fulfill (iv) 0 < r(a) < T(a) < d(a) - 1 for any a g A, cf. [9]. Also, instead of m machines 
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being available over time there might be specified a so-called m-profil p, which gives for all time 
intervals [i, i+1), i = 0,l,2,...,t - 1 the number p(i) < m of available machines and which 
corresponds to a more restrictive version of the contraints (iii), viz. (v): |{ae A | T(a) = i} | < p(i) 
for all i. 

It is interesting to note that a polynomial time algorithm for the given normalized 
version of the m-machine unit time scheduling problem (m arbitrary, but fixed) yields, in fact, a 
polynomial time solution to the very general problem formulation (i), (ii), (iii), (iv), just given. This 
is obtained by the following. 



GENERAL NORMALIZATION PROCEDURE: 

Let a general m-machine unit time scheduling problem be given by a partial order (A,0), a time limit 

t-1 

t, release dates r and due dates d and, finally, an m-profil p. Let k: = £ p(i) and assume |A| < k. 

i=0 

Construct the (m+l)-machine problem [(A', O'), t], where |A'| = (m + l)*t and where (A', O') is 
obtained from (A,0) as follows: 

1. add an independent chain of new elements PpP 2 ,--,P t ; 

2 . add k - 1 A| independent elements ; 

3. add for each time i = 0,1,..., t - 1 m - p(i) new jobs that are independent from all original 

jobs and the jobs introduced in (2), and that have release dates i and due dates i+1, 

respectively. 

4. add a new ordering relation (p s , a) whenever a job a (including those introduced in (3)), 

has a release date i > s and a new ordering relation (a, p s ) whenever a job a (including 

those introduced in (3)) has a due date i < s. 

The idea behind the construction is as follows. Every schedule completing the modified problem in t 
time units on m + 1 machine will w.l.o.g. have to process the chain introduced in (1) on a particu- 
lar machine, with job pj being processed exactly in the time interval [i - 1, i). By the construction 
given in (3) and (4), exactly m - p(i) artificial jobs will also be executed in the time interval [i - 1, i). 
Thus exactly p(i) time slots will be left in this interval to schedule the original jobs. Again because 
of (4), original jobs can, because of the newly introduced ordering relations in (4), be only 
scheduled in accordance with their original release and due dates. Finally, the k - |A| independent 
jobs, introduced in (2) may fill the not used time slots in any solution that solves [(A, 0), t] for the 
given m-profil. Using these properties, it is easy to establish that any feasible schedule for the 
original problem extends to a solution for the normalization, while a restriction to the set A of any 
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feasible schedule for the normalization is a feasible schedule for the original problem. The transfor- 
mation is obviously polynomial (even linear, except for the increase of the number of machines) 
and therefore settles the problem. Note, however, that for special classes of posets, the construction 
may again leave the special class. 

The two normalizations discussed are a good motivation to study particularly the problem type 
[(A,0),t] with |A| = m-t on m machines, as already proposed in [1]. The question for the 
existence of a feasible schedule is then equivalent to asking, whether all m machines can be utilized 
uninterruptedly in the time interval [0,t]. This formulation implies that if there is no such schedule, 
then - regardless of how we schedule - we will at some time i = 0,l,2,...,t - 1 end up with the 
following situation: 

All machines were utilized uninterruptedly until time i while at time i, we can schedule at most 
r < m - 1 0-independent jobs Yi,.-*>Y r (where w.l.o.g. m > 1). Also, all remaining jobs are 
obviously successors of at least one Y s > s = l,...,r, for otherwise, an order-minimal job (i.e. a job 

without predecessors) among the remaining jobs could be scheduled at time i in addition to jobs 

r 

Yiv>Yr ThusD, the set of the remaining jobs can be represented as D = kJ N(Yj), where N(a) 

j=l J 

denotes the set of successors of a job a. Note that it is impossible to schedule all jobs in D in 

the remaining time interval [i + 1, t). This is evident because of |D| / m > t - i; in fact, we have 
|D| = nr(t - i) + (m - r). Note that we might determine in advance for any of the at most 0(n m ) 

many 0-independent sets B of jobs of cardinality |B| = r<m the cardinality | kJ N(y) | of 

ysB 

their successors and, in any case (or just in case | kJ N(y) | = s*m + (m-r) for some s e N) 

jeB 

establish t - (s + 1) as an upper bound b*(B) on the completion of the set B in a feasible sche- 
dule. Here, b*(B) being an upper bound on the completion of B is to be understood as the 
requirement that at least one job from B must be completed by time b*(B). 

The given bound is imposed right away on a particular job a if B = {a}; we write b*(a) in this 
case. As this is the only case occuring in 2-machine problems, this observation will later lead to a 
well-known polynomial time procedure for this case [8]. As soon as m > 2, for 0-independent sets 
B with two or more jobs, it will not be clear in general, which of the jobs in B should have to 
satisfy the obtained bound. If not for special reasons one of the jobs will have to be completed 
before the obtained bound anyway, we arrive at a choice problem, where the number of all 
combinations of choices will, in general, grow exponentially in the number of jobs. This is 
what makes the m-machine case for m > 2 so much more involved than the case of just 2 machines. 
The major conjecture in Section 4, however, will be that choices have to be taken into consideration 
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only in those cases in which only a particular job from the set B is able to accept the bound b*(B) 
in a feasible schedule, and that these cases can be identified in polynomial time. The following 
example will illustrate some of these aspects: 

EXAMPLE 1: 

Consider the following (3-critical; see below) poset 0 = (A,0) with A = {1,2,..., 12}, given in 
Figure 1 by a node-diagram, cf. also [26]. We ask, whether 0 can be scheduled on 3 machines in 4 
time units (which turns out to be impossible). 




Fig. 1 : A 3-critical poset 



(1) Consider the partial schedule, where we start with jobs {1,2,3}, continue with {4,5,6} and 
then are stuck with jobs {7,8}. Here, job 8 has four successors, thus an individual bound 
b*(8) = 4-2 = 2 and thus should have been completed earlier, i.e. by time 2. 

(2) Consider therefore the alternative consisting in starting jobs {2,3,4} and then leading to 
the bottleneck {1,8}. Note that here both jobs have fulfilled there individual bound which is 3 
for job 1 and 2 for job 8. Note also, that all three jobs 2, 3 and 4 have the individual bound 1. 
Now consider again the set {1, 8}. In total, both jobs have 7 successors, i.e. we have 
b*({ 1,8}) = 4-3 = 1. This means that one of the two jobs must be completed by time 1. 
This is impossible for job 8, as it has predecessors. Therefore, we obtain b*(l) = 1. However, 
given that jobs 2, 3 and 4 have the same individual bound, we obtain a contradiction, i.e. 0 
cannot be scheduled on 3 machines in 4 time units. 
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The individual bound b*(oc), oc e A, just introduced, can in fact be tightened in form of a 
bound b fulfilling b(a) < b*(a) for all a e A, using a backward induction. Example 1 gives a 
motivation for that. Consider again job 2, which has 5 successors. This could be used to obtain the 
immediate bound b*(2) = 4 - 2 = 2 for this job. However, with job 8 being a successor of job 2 and 
b*(2) = 2, the sharper bound b(2) = 1 used in Example 1 (2) is obtained. 

So, in general, we will define a sharper bound b recursively (cf. [8]), starting from maximal 
elements p of 0 (for which b(P) = t), and then put for non-maximal elements a 

b(a) = min {i - [~|{Pe A | pe N(a) and b(P) < i} | /ml }, 
ieC 

where C is the set of all time points in {l,...,t}, such that there is a successor of a that has to be 
completed by that time. (Note that always t e C). I.e. for all time points i, that are upper bounds for 
the completion time of certain successors of a, one has to take care, that the time interval [b(a), i) is 
long enough to handle all the respective jobs on m machines. The earlier bound b*(oc) is contained 
here in the minimization process for the case i = t. 

It is immediate that no schedule can exist, as soon as for some time i, the number |{aeA| 
b(a)<i}| strictly exceeds nri; cf. also the argument used in Example 1(2), where |{oceA|b(a) 
< 1 }| >3*1 was fulfilled. In fact, the converse is also true for the 2-machine case, i.e. there is a 
feasible schedule, as soon as the condition |{ae A | b(a) < i} < nri is fulfilled for all i = 
0,l,...,t - 1. This is actually the basis for known polynomial time algorithms for the 2-machine case 
[8] and will prove to be a special case of the basic representation theorem, following next. 
This representation theorem will extend the 2-machine case approach to the general m machine unit 
time scheduling problem. To achieve this, we will have to replace b*(B) by the sharper bound b(a), 
obtained by backward induction, also in case of |B| > 1. The main obstacle is that we do not 
immediately know, how to handle bounds of this type at all, i.e. what to do with them, how to 
adapt them and how to relate them to certain jobs in the respective sets B. Our approach will 
intuitively be to focus at the following aspect. Given any feasible schedule for the problem under 
consideration and given any correct bound deducible for a 0-independent set B with 1 < |B| < m 
that is consistent with the given schedule, there will be at least one element a in B that observes 
this bound. Now pick a particular such element Oig from each such set B and call the function 
B — » (Xg (for all B considered) a choice. Note that , intuitively speaking, (Xg being the choice for 
B means that the bound for ocg, defined next, will have to take into account all sucessors of 
elements in B. In order to do this, the bounds must already been defined for all these successors. 
It may happen that, given a particular choice, some elements can block each other in the sense that 
there is a circular dependency among them concerning the need of knowing already their 
respective bounds (cf. a simular phenomena in connection with strategies, induced by so-called 
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selections [15], [16] in stochastic scheduling models). We call a choice consistent or 
non-blocking , if such a circular dependency does not exist. This is equivalent to the existence of 
a sequence among the jobs that allows a backward recursion as needed in the following 
definition of a choice dependent bound for non-blocking choices that is analogous to the 
individual bound discussed above: 

(1) Put b(a) = t for all maximal elements a 

(2) Let a be non-maximal; a is the chosen element for certain considered 0-independent sets B, 
in particular a is chosen for the set B = {a}. 

Put then b(a): = min (b(B,a) | a chosen for B) where 

b(B,a): = min {i-fl{pG LJ N(y) and b(p) < i} | /ml}. 
i€C(B) yeB 

Here, C(B) is the set of all time points i in {l,...,t} for which there is a successor of some 
y g B having an individual time bound smaller or equal to i. Note that w.r.t. all b(B,a), a chosen 

forB, KJ N(y) will be non-empty, as there will be successors of a. Also, the bounds for all jobs 
yeB 

in this set will already be determined because of the choice being non-blocking and the sequencing 

within the backward recursion was chosen accordingly, and, finally, all P g LJ N(y) will 

ycB 

certainly fulfill b(P) < t, i.e. t g C(B) will be valid for all considered sets B. Obviously, the 
given bound is sharper than the earlier bound computed on the basis of individual elements only 
and also b(B,a) < b*(B) is obvious. 

As was already mentioned above, an easy inductive argument shows the following: starting 
from a feasible schedule T, a consistent choice for all considered sets B can be made such that 
the given schedule observes the corresponding choice dependent bound, i.e. such that T(ocg) + 1 < 
b(ag) < b(ag, B) is fullfilled for any considered B and any chosen ocg. To achieve this, one 
merely has to chose for any B an element within this set with earliest starting time w.r.t. T. This 
consistency of T with the bounds b means, in particular, that w.r.t. these bounds, a contradiction of 
the type |{ocg A | b(a) < i} | > m*i for i g {0,1,... ,t - 1} can again not occur. 

The following theorem now shows that again the converse is also true: 



BASIC REPRESENTATION THEOREM 

Let [(A,0), t] denote an m-machine unit time scheduling problem fulfilling |A| = m*t. Then the 
following holds: 
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There is a feasible schedule for the problem, iff there is a non-blocking choice ocg for 
each 0-independent set B of jobs with 1 < |B| < m, such that w.r.t. the correspondent bounds b, 
there is no contradiction of the form |{ae A | b(a) < i} | > nri for some i e {0,l,...,t-l}. 

If there is a feasible schedule, it may e.g. be constructed by successively starting in the i-th. 
time slot, i = 1,2,. ..,t any set of m jobs among the respective set of minimal jobs remaining, such 
that w.r.t. the bounds bj(oc): = b(a) + 1 - i on the remaining jobs, there is no bound contradiction 
as considered. Suitable sets of jobs will be available in any time slot. This means, in particular, that 
we can do priority scheduling w.r.t. the priority list induced by increasing values of b. 



Proof: 

The argument for the easy direction was already given above. So assume there is a non-blocking 
choice ocg for all 0-independent sets B with 1 < |B| < m, such that there is no bound contradiction 
|{ae A | b(|3) < i} | > m-i for i e {0,l,...,t - 1} w.r.t. the correspondent bounds b(a), a e A, 
defined above. We then have to show that there is indeed a feasible schedule for this problem. The 
proof is given by induction on t, where the case t = 1 is trivial. So assume the result to be true for a 
certain value of t and consider the case of time limit t + 1 and nr(t + 1) elements. 

Assume first that the set M of minimal elements in 0 has strictly less than m elements and let 
a m e M denote the chosen element within M. Obviously, we obtain b(aj^) < b(M, Ofyj) < t - 1 = 
0, and thus |{cxe A | b(a) < 0} | > 0, a contradiction. 

Consequently, we can find a set S of m elements to start with. Note that after removing any 
such m elements, we come to a subproblem with a base set A", where the question is to schedule 
the remaining nvt elements in t time units on m machines. For this subproblem, the induction 
hypothesis can be applied. We try this w.r.t. the restriction of the original choices made for jobs 
and sets of jobs within the subproblem. If we compute the bounds b' for the subproblem and take 
into account the backward recursion, it is easily seen that these are greater or equal to the original 
bounds b, reduced by 1, i.e. we have b'(oc) > b(a) - 1 for all a e A". Here, the reduction by one is 
due to fact that the recursion in the subcase starts from b(p) = t instead from b(p) = t + 1 for 
maximal elements and is otherwise basicly identical, while a possible increase comes inductively 
from the elemination of those considered sets B that contain some of the unstarted elements but had, 
as chosen element ocg, an element belonging to A'. Assume now, there is no bound contradiction 
of the form |{ae A" | b'(a) < i}| > i*m for i e {0,1,. ..,t - 1} for the subproblem. Then by 
induction, we can schedule the jobs in A" in t time units, leading to the desired schedule of all jobs 
in A in t + 1 time units. Thus we are done, whenever we can start with m elements that avoid a 
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bound contradiction for the remaining problem. We will show in the following that at least one such 
start set always exist, viz. a set S of jobs with the lexicographically smallest (sharpest) 
bounds. In fact, we will even show that taking away this set avoids even any contradiction for the 
subproblem w.r.t. the sharper bounds b(a) - 1, a e A". Thus the proof allows us to schedule jobs 
according to the priority list, induced by the originally computed bounds b(a), a g A, as stated 
in the theorem. 

Assume that this is not true, i.e. there is a bound contradiction \{ae A' | b(a) - 1 < i}| > i*m 
for some i e {0,1,..., t - 1}. Certainly, then {ae A" | b(a) - 1 < i) c {ae A | b(a) < i + 1}, where 
i + 1 e {0,1,. ..,t}. If we assume that b(p) < i + 1 is true for all jobs in the starting set, we obtain a 
bounding contradiction for the original problem. Therefore, there is a proper subset B with |B| < m 
among the started jobs (i.e. B cS), which contains exactly those started elements p, fulfilling b(p) 
< i + 1. B is not empty, as an element with the smallest bound was started and necessarily belongs 
to B. Let ccg denote the chosen element in B. Then we obtain b(ocg) < b(B,ag) = (i+ 1) - (i + 1) = 
0 because of the immediate fact that all elements from {ae A" | b(a) - 1 < i) must be successors 
of elements in B. This follows, as none of these element can be minimal in 0, due to the fact that 
jobs with greater bounds (even bounds greater than i + 1) belong to the lexicographically smallest 
starting set, viz. are the elements of the set S \ B. Furthermore, these jobs from S \ B cannot be 
predecessors of jobs p fulfilling b(p) < i + 1 because of monotonicity reasons w.r.t. the bounds 
(i.e. by construction, the bound of a predecessor has always to be smaller than the bound of a 
successor). Therefore, we indeed obtain the inequality b(ag) < 0 which constitutes a bound 
contradiction, concluding the proof. ■ 



The proof shows that bounds b(B) for 0-independent sets B of jobs with 1 < |B| < m must 
only be determined if a situation occurs in which, for a certain time bound i, all jobs fulfilling 

b(a) < i belong to the set B u LJ N(y) and, in addition, fulfill | LJ N(y) | /m > i - 1. Asa con- 

ye B ye B 

sequence of this observation, the consideration of choices for the computation of the respective 
bounds can be restricted to a subset of all sets B considered so far, whenever contradictions of the 
type considered can only happen for sets B within this subset; cf. in particular Lemma 4. I.e. the 
Representation Theorem remains valid for the bounds b computed on the basis of such a restricted 
set of considered sets B with 1 < |B| < m. This is the statement of the following Corollary. 
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COROLLARY 2: 

Let M denote a subset of the set of all 0-independent sets B of jobs with 1 < |B| < m, such 

that contradictions of the form [b(a)^i for some i=l,...,t implies aeBuUNfy) 

yeB 

and N(y) | /m > i - 1] can only occur for sets B e M . Then the Represenation Theorem 
yeB 

remains valid, if choices and bound computations are restricted to the consideration of sets in M 
only. 

In the following, we briefly discuss the consequences of the Representation Theorem for an 
algorithmic treatment of the problem. In this context, we first observe that (w.r.t. the number 
n of jobs) 0(n m ) is about the best complexity bound one can probably hope for in the general 
m-machine case. For 2 machines, this is the exact bound in the known solution [8]. Furthermore, 
the 2-machine case with release and due dates, which can be regarded as a special version of the 
3-machine case via the general normalization procedure given, requires an O(n^) time bound in the 
known solution [9]. Annother motivation for 0(n m ) as a lower bound on the needed computation 
time in the apparent need to consider all independent sets of jobs with cardinality up to m - 1, 
which are 0(n m ‘^) many, and the need to do some involved computations w.r.t. all these sets. 

We will discuss now that in a certain sense then, a solution for an m-machine problem could 
essentially be interpreted as the task of finding a test-choice for such a problem, i.e. to 
determine a choice with the following property: either there is a schedule compatible with the 
choice, or there is no feasible schedule at all. If this can be achieved in 0(n s ) time with s > m 
arbitrary, the general m-machine problem can be solved in 0(n s ) time. I.e. all the other steps arising 
from the Representation Theorem, such as testing the consistency of the given choice, 
computation of the associate bounds, testing the considered bound condition and, finally, 
producing a feasible schedule can be done in 0(n m ) time, m > 1 arbitrary. As an application, 
we will demonstrate in Section 3, how some of the solved special case of m-machine unit time 
scheduling can, in fact, be obtained via a solution of the test-choice problem, followed by an 
application of the Representation Theorem. 



CHOICE VERIFICATION ALGORITHM 

Given a m-macliine unit time scheduling problem [(A, 0), t] with A = m-t, a set M of 0-inde- 
pendent sets of jobs B with 1 < |B| < m as required for consideration, and an arbitrary choice B — » 
ag for all B e M , it is possible to check in 0(n m ) time whether there exists a valid schedule 
compatible with this choice and to produce such a schedule, if it exists. In particular, checking 
includes the following steps: 
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1. Checking for m>2 in 0(n m ) time, whether the given choice is non-blocking, and if it is 
not, determining a valid sequence for the backward recursion. 

2. Computing the choice-dependent bound in 0(n m ) time, using the obtained sequence for the 
backward recursion, if the choice turned out to be non-blocking, 

3. Checking in 0(n 2 ) time, whether there is a bound contradiction of the form |{ae A | b(a) < 
i}| > m*i for some i e {0,1,2,..., t - 1}, 

4. Producing in O(n^) time a feasible schedule compatible with the given choice (in case there 
was no bound contradiction) by doing priority scheduling w.r.t. the obtained bounds b. 



We shortly argue, how the described steps have to be organized in order to obtain the given 
time bounds. The validity of the whole procedure is of course due to the Representation Theorem. 

We start from a choice being given in the form of a list B -> ocg for all B e M, where | M | is 

bounded by 0(n m_1 ). In 0(n m ) time, we can, using bucket sort, reorder this list into the form 
ocj, M j, 0 C 2 ,M 2 ,..., ttn-i^n-l’^^m w ^ere the sequence of jobs is arbitrary and where M j 
gives the (possibly empty) sets in M , for which cq is the choice. Note that the whole list has 
length 0(n m "l), while each |M j| is bounded by 0(n m '^). 

Also, as part of the data structures involved, we keep a list of the present bound values b(a) 
for all a e A, with b(a) = t as common start value. We will also maintain a modified list in which 
time points i = 1,2,3, 4,. ..,t are followed by those jobs which currently have the bound i. Note, 
finally, that a major step will be the determination of the union of two sets of successors in a 
particular form (merging of sorted lists with preprocessing via bucket sorts). This union 
operator will need linear time and will be defined as follows: successor jobs are arranged in 
advance in increasing order concerning the given numbering of jobs. Jobs are picked up one by one 
(at most n many) from each of the two involved successor sets, their present bound is taken from 
the current bound list and the element is then put into a list (if not already there) of jobs in the union, 
ordered as follows: after each time point i those jobs with present bound i follow in increasing 
order. 

Step 1: 

Given the preparatory data organisation, we check for m > 2, whether the choice is non-blocking 
(blocking is impossible for m < 2). In order to do this, we determine the relation R on A defined by 
(a, p) e R: <=> [p e N(y) for some ye B, B e M , and a is chosen for B]. I.e. (a, p) e R 
iff the bound for p must be known in order to compute the bound for a. R is obtained by applying 
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for all a and all sets B e M a the union operation, consuming linear time. I.e. for each a, at 
most m times 0(n m ~2) applications will be performed. This leads to an 0(n m ) time complexity in 
order to determine R. It is easily verified that the considered choice is non-blocking iff R is 
acyclic . This property of R can be checked by standard methods in O(n^) time. If R is acyclic, 
then obviously, the dual of any linear extension of R is appropriate as a sequence for the 
backward recursion. Such a sequence can again be determined in O(n^) time by standard methods. 

Step 2: 

We determine the bounds b, starting from the original modified list of at most 0(n m "^) considered 
sets B. In doing so, we use the sequence among the jobs determined in Step 1 for the backward 
recursion. What we essentially have to achieve is to determine in linear time for any considered 
set B, a being the choice for B, the number 

b(B, a): = min {i - [ | {(3 e U N(y) and b(y) < i} | /ml }. 
ye C(B) ye B 

It is easily seen that, once this is achieved, b(a) itself is obtained in 0(n m “^) time (remember 
| M a \ < 0(n m '2)) and thus b is achieved in 0(n m ) time, as stated. 



Now it is easily seen, that the determination of b(B,a) involves m applications of the linear 
time union operator described above, where the operator itself makes use of the available list 
of all (required) present bounds. This list will be updated with b(a), once b(a) is determined. 

After having applied the union operator m times (m fixed), we go through the resulting list of jobs 
in linear time. We will see for each time point i = l,...,t, in increasing order, whether there are 

jobs in LJ N(y) accepting this bound (while not accepting the previously considered smaller bounds 
yeB 

bounds), we count the number of such jobs, add this number to the total number of jobs found 
already earlier for time bounds up to i - 1 (where a sum greater to zero exactly means i e C(B)), 
compute the time bound associate with i, update the minimum over all such bounds up to this 
moment, and proceed to i + 1. This essentially concludes Step 2. 

Step 3 and 4: 

These two steps are not time-criticial and can be done in a straightforward way. 

In summary, the Representation Theorem delivers an algorithmic tool which, in 0(n m ) time, 
allows us to check whether any particular conflict solving mechanism on sets of jobs that 
might eventually turn out to be bottlenecks within a schedule, really works. The use of such a test 
device for special problem instances as well as for the general case will be the subject of Sections 3 
and 4. 
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3. SOME WELL-BEHAVED SPECIAL CASES 

In this section, we give five applications of the Representation Theorem to well-behaved special 
cases of m-machine unit time scheduling problems. These cases are the 2-machine case (Lemma 3), 
a special 3-machine case (Lemma 4), a scheme for the general incorporation of release and due dates 
(Lemma 6), interval orders (Lemma 7) and forests (Lemma 8). 



LEMMA 3: (cf. also [2], [6], [7], [8], [9]) 

2-machine unit time scheduling is possible in O(n^) time. 



Proof: 

As argued in Section 2, the Representation Theorem applies to the normalized version of the 
2-machine case without any choice problem at all. Thus Steps 2. - 4. of the choice verification 
algorithm solve the problem in O(n^) time, i.e. either establish inconsistency of the problem or give 
a feasible schedule. The time bound is the best worst case bound known w.r.t. the number of jobs 
(cf. [6], [8]), in fact, the algorithm coincides with the algorithm given in [8]. For a somewhat 
improved algorithmic approach cf. [7]. m 



LEMMA 4: (cf. [9]) 

A 3-machine unit time scheduling problem [(A,0), t], with (A,0) containing a chain pj, P 2 , ... , P t 
of length t, can be solved in O(n^) time. [Note that via the general normalization procedure given, 
|A| = m*t can be assumed w.l.o.g. and also, any 2-machine unit time scheduling problem with 
release and due dates can be brought into this form, cf. also [9]. 

Proof: 

The proof consists in three major steps: first we identify (implicitly) an appropriate system M of 
sets to be considered, then we give an associated test-choice and, finally, we argue how to modify 
the choice verification algorithm in order to cope with this implicit characterization of the system M 
(and the corresponding test choice) in a way that does not increase the time complexity O(n^). 
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Step 1: Definition of M : 

We start from the observation that if the given problem has no solution then any possible 
fixing of M and any choice w.r.t. M will lead to a bound contradiction because of the Representa- 
tion Theorem. If instead there is a solution, then w.r.t. any successful choice, job Pj from the given 
chain of length t is scheduled in the time interval [i - 1, i], i = l,...,t. (Also, P^ has no predecessors 
and p t no successorss). These facts impose an a priori bound of b(y): = min{i g {l,...,t - 1} | 
Pj +1 g N(y)} on all jobs ye A fulfilling (y,p-) € 0 for some i = 2,...,t. The respective elements 
and bounds can be determined in 0(n^)-time beforehand. Also it can be checked whether {a g 
A | b(a) < i} < 34 is fulfilled for all i = 0,l,...,t - 1 w.r.t. the a priori bound. If not, the problem 
has no solution. If the condition is fulfilled, we ask which sets B of 0-independent jobs with 1 < 
|B| < 2 have to belong to M in order to allow an application of the Representation Theorem in the 
sense of Corollary 2. The inductive argument from the proof of the Representation Theorem shows 
that exactly those 0-independent sets B with 1 < |B| < 2 have to be considered for which, after 
having used all machines uninterruptedly in an appropriate way up to time j - 1, B becomes a 
bottleneck in the time interval [j - l,j) in the sense that w.r.t. the intended bound b, the condition 
|{8 g kJ N(y)| b(8) < i} | /3 < i - j for some i = j + l,...,t is violated. We implicitely define M 

YgB 

to be just the set of all those potential bottleneck sets B. Obviously, all these sets B are either of the 
form {pj} for j = l,2,...,t - 1 or of the form {pj,a}, j = l,...,t with a 0-independent of Pj and 
b(a) > j. Unfortunately, we do not know in advance which of the given sets really belong to M . So 
we have an additional problem to cope with when discussing the algorithmic treatment in Step 3. 



Step 2: The test choice for M 

The choice for a set {pj} g M is of course pj, while for sets (Pj, a} g M it is a. This is, 
in fact, the only potentially successful choice at all, as any other choice for a set {pj,cx} g M 
would imply b(pj) < j - 1 and thus b(pj) < 0, which yields an immediate bound contradiction. (The 
given choice is non-blocking; the dual of any linear extension of (A,0) could be used for a 
backward induction). 



Step 3: 

A variant of the choice verification algorithm for the given, implicitly defined set system M . 

(1) We start with the described a priori bound for jobs that are predecessor of some pj, j = 2,...,t 
and with b(a) = t for all other jobs. For maximal elements of (A,0), the bound b(a) = t will never 
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change. For all other jobs, the bound will be updated from time to time, always keeping the least 
value obtained so far. The main idea is to scan the sets B of type {Pj}, j = l,...,t - 1 and {a, Pj}, j 
= 1 t, that may belong to M . The aim is to identify, whether such a set really belongs to M , i.e. 

whether it leads to a bottleneck of the form |{8 e kj N(y) | b(8) < i} | /3 > i - j. As soon as 

yeB 

the bottleneck-situation is identified, the corresponding bound i - [ |{8 e LJ N(y) | b(8) < i}| /3l 
is imposed on a in the updating process. 

(2) The main problem in (1) is how to cope with the problem of having to rely on knowledge about 
b(8) < i, which itself depends on the previous identification of (parts of) M. A solution to this 
difficulty is obtained by the following observation. In order for B = {a,Pj} to be a member of M , 

a contradiction of the form |{8 e N(y) | b(8) < i} | /m > j - i for i e (j + l,...,t} is needed, 

yeB 

where, in addition j < b(a) < i. Here, b(a) < i arises from the fact that if no successor 8 of a 
fulfills b(8) < i, b(Pj) < j - 1 would follow, leading to the bound contradiction b(p j) < 0. 
Certainly, if we would know in the above situation the bounds for all successors of a and pj, we 
could check, whether there is a bound contradiction for M . If it occurs, this would result in an 
updating of b(a), leading to a value b(a) < j - 1. 

The intended procedure will make use of this observation and needs the notion of the dual 
height h(a) of a job a w.r.t. (A,0), where h(a) denotes the cardinality of a maximal chain in 
(A,0), starting from a. The dual height of all jobs can be computed beforehand by standard 
methods in O(n^) time. We can check within the same time bound whether the dual height of each 
Pj is t - i + 1, i = l,...,t. If this is not the case, there is no solution. If instead it is true, we know 
that the dual height of all successors of pj must be less or equal to t - i. 

Notice now that for all elements of dual height 1, i.e. for all maximal elements 8, the bound 
b(8) = t is correct and will never change. This allows us to check all sets {p t ,a} for all a e A 
with h(a) = 2. In all cases this will lead to a new bound b(a) < t. We next check B = {p t _j}. If 
this yields the necessity to decrease b(p t . j), the problem has no solution. Otherwise, we consider 
sets {p t .j,a} with h(a) = 2 in those cases, where b(a) = t - 1. This may lead to a smaller bound 
b(a) < t - 1. We proceed with sets {Pj^}, i = t - 2, t - 3,...,1 and jobs a of dual height 2 with 
present bound b(a) > i, always checking before the set B = {pj} w.r.t. inner time bound t, to see, 
whether there is no contradiction in order to proceed.The bound computation w.r.t. {pj,a} is also 
restricted to the inner bound time t (i.e. to the case i = t in the given formula) and needs linear 
time, cf. the choice verification algorithm, i.e. we only need to see whether |N(a) u N(pj) | /3 > t 
- i in order to decide, whether {pj,a} e M . Computing, if required, the associated bounds, we 
finally arrive at the correct bound b(a) for a, which in turn, will be used to update the bound of 
predecessors of a by comparison with b(a) - 1. Le. to summarize, we can compute the correct 
bound for all jobs of dual height 2 because w.r.t. their successors 8, which are all maximal, the 
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bounds b(8) = t will not change. Consequently, computations are restricted to inner bounds t only. 
Note that after this step, the bounds for all jobs of dual height smaller or equal to 2 will be known. 
Also note, that sets (P-,a) were considered at most once. 

If we proceed now to jobs of dual height 3, there will (after a positive check for (Pt-2^ anc * 
inner time points i = t - 1, t) occur the additional difficulty that such a job a may have a successor 
with a bound b(a) = i, i < t - 2, i.e. a current bound of b(a) <i-l<t-2. If we now would try to 
consider B = {pj. j,a}, the needed bounds of certain successors of pj_ ^ might still be missing. We 
therefore will treat only those jobs of dual height 3, whose present bound b is equal to t - 2 and 
postpone jobs with present bound t - i, i > 2, until the consideration of jobs of dual height i + 1 > 
3; note that the bound of jobs of dual height 3 is necessarily less or equal to t - 2. This procedure 
then leads to the following general step. 

Assume we have scanned iteratively all jobs of dual height l,2,3,...,r. In doing so, a job a 
from this set either obtained his correct bound b(a) > t - r + 1, or it has a current bound b(a) < t - 
r. We then go over to r + 1. This means that we first check for the set B = {p t _ r }, whether it 
necessitates a decrease of the bound of P t _ p which means that there is no solution to the problem. 
Note that all successors of P t _ r were already scanned. If any of these successors 5 has a present 
bound b(S) < t - r, the problem would have again no solution. Otherwise, the bounds of all 
successors are correct and the required computations can be done for all inner time bounds i = t - r 
+ 1, t - r + 2,...,t simultaneously in linear time (cf. the choice verification algorithm). Again, if 
the bound of P t _ r should be decreased, the problem has no solution. Otherwise, the bound b(p t _ r ) 
= t - r and the bounds b(8) > t - r + 1 of all successors of p t _ r are correct. 

In this case, we now consider for j = t - r, t - r - 1,...,2 sets B = {a,Pj} for all those a e A, 
for which the previous bound b(a) equals t - r. In doing so, we consider (only) the inner time 
points t = t - r + l,...,t, because a has no successors with a bound smaller or equal to t - r. We 
scan those jobs in a sequence of increasing dual height, where the dual heights involved maybe 
2,3,...,r + 1. Note that once a (first) set (Pj,a) e M is identified, it leads to a present bound 
b(a) < t - j - 1 for a and b(8) < t - j - 2 for all predecessors of a. (Thus, in particular, all elements 
8 of dual height r + 1 have a bound b(8) < t - r). If the bound of a was decreased, we postpone the 
further treatment of a until we get to treatment of the respective greater dual height). To see the 
correctness of the procedure one only has to observe, that if a set B = {a,pj} with j < t - r is 
considered, we need only take into account inner time bounds i = t - r + 1, t - r + 2,...,t (as a has 
no successor 8 fulfilling b(8) < t - r). Now while the bounds for elements in N(a) are already 
known, we know that all those, elements 8 in N(Pj) whose bound is not known certainly fulfill 
b(8) < t - r + 1. This is obviously enough to carry out the required computations. Again, as 
demonstrated in connection with the choice verification algorithm, all computations needed in 
connection with B can be performed in linear time. If, in this process, no bound contradiction is 
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obtained, the correct bound b(a) = t - r of a is identified, i.e. this value can never change again. 
Otherwise, we obtain for some first j < t -r a bound b(a) < t - r - 1 and will consider the job again, 
later, as described above. Note that here again, sets (Pj,a} are only considered once. For, either 
there will be no bound decrease for a for all j < t - r, in which case the correct bound of a was 
obtained, or else, there is a first j that leads to a bound decrease. In this case, we will later on 
continue with sets {Pj,a} where j < j'. This concludes the modified choice verification algorithm. 

(3) W.r.t. the needed computation time, one realizes that in a certain sense the given procedure is a 
kind of reordering of the original choice- verification algorithm. The potentially O(n^) many 
elements of M are looked through in a different order, not partitioned into blocks depending on the 
chosen a. The modification was however done in a way that still achieves the time complexity 
O(n^). The main feature to this aim is the already demonstrated fact, that each of the n sets {pj} is 
considered once, and that similarly each of the at most O(n^) many considered sets (Pj,a) is taken 
into account at most once , where the computational effort in connection with each considered 
set is linear. This yields the stated complexity bound O(n^). m 

EXAMPLE 5: 

Consider the 3-machine unit time scheduling problem [(A,0), t] with A = { 1 21 }, 0 as given in 

Figure 2 and t = 7. Note that jobs 1,2,. ..,7 constitute a chain of length 7 in (A, 0). 

(1) We run the modified choice verification algorithm for this problem as described in the proof 
of Lemma 3 and demonstrate that there is no feasible schedule for this problem. 

1. The a priori bounds are b(i) = i for jobs i = 1,...,7. 

2. For the maximal elements 7, 15 and 21, the correct bound is 7. 

3. For the jobs of dual height 2, i.e. for jobs 6, 14 and 20, the bound 6 is obtained w.r.t. the 
inner time bound 7 and all elements i = 1,...,6. Therefore 6 is the correct bound for these 
elements. 

4. For the jobs of dual height 3, i.e. for jobs 5, 8, 9, 10, 11, 12, 13 and 19, the bound 5 is 
obtained w.r.t. the inner time bounds 6 and 7 and all elements i = 1,...,5. Therefore, 5 is the 
correct bound for these elements. 

5. For the jobs of dual height 4, i.e. for jobs 4, 16, 17 and 18, the following happens. For job 4, 
the correct bound 4 is obtained. For jobs 16, 17 and 18, the consideration of the sets {2, 16}, 
{2,17} and {2,18} yields, for any of the involved inner time bounds 5, 6 and 7, the value 1. 
I.e. for jobs 16, 17 and 18, the associated bound will be smaller or equal to 1. As b(l) < 1 is 
true due to the established a priori bound, the bound contradiction {a e A | b(a) < 1} > 3 is 
obtained. This means that there is no feasible schedule for this problem. 
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(2) We relax (A,0) in Figure 2 by eleminating the relation (18, 19), i.e. job 18 will become an 
immediate predecessor of job 20. In this case, 18 is an element of dual height 3 and the 
corresponding bound will be 5. The correct bounds for jobs 16 and 17 would finally be 1 and the 
bound b(i) = i for i = 1,...,3 would also be confirmed. So, we would end up with the following 
situation: 

b(l) = b(16) = b(17) = 1, b(2) = 2, b(3) = 3, b(4) = 4, b(5) = b(8) = b(9) = b(10) = b(ll) = b(12) 
= b(13) = b(18) = b(19) = 5, b(6) = b(14) = b(20) = 6 and b(7) = b(15) = b(21) = 7. Obviously, 
there is no bound contradiction, i.e. there is a solution to this scheduling problem. A valid schedule 
corresponding to the given bounds is given by its Gantt-chart in Figure 3. Note that the schedule is 
uniquely determined up to permutations among jobs 8,.. .,13. 




Fig. 3: A solution to a slight modification of the problem discussed in (1), represented 

by its Gantt-chart 
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We just mention that Lemma 4 (cf. [9]) yields the polynomial time solvability for 3 machine 
unit time scheduling problems [(A,0), t] for which (A,0) contains a chain of length greater or equal 
to t. This can be slightly extended to the existence of a chain of length t - r, r fixed, in (A,0). For an 
0(n 3+2r ) algorithm to solve this type of problems cf. [1], [23]. 

Lemma 4 also shows a way to extend a solvable special case to the additional consideration of 
release and due dates. The extension for an m-machine problem [(A,0), t] would be based on the 
generalized normalization procedure, i.e. means the transition to an (m + 1) machine problem 
[(A", O'), t] for which (A', O') contains a newly introduced chain that also serves as a 

means for the incorporation of release and due dates via order relations. The basic idea would again 
be to restrict M to those sets B with 1 < |B| < m + 1 that contain an element (3j from the chain and 
that induce a bound smaller or equal to j - 1. In this situation, the ability to identify a test choice 
among the (at most m - 1 many) jobs different from Pj in B would be sufficient to yield a solution 
to the more general case involving release and due dates. I.e. a general polynomial solution for 
finding test choices for sets of at most m - 1 jobs (m machine case) leads to a solution for the 
m-machine case with release and due dates along the lines described in connection with Lemma 3. 
To put this more formally, we have the following: 



LEMMA 6: 

Any general 0(m r ) test choice construction procedure for sets of jobs of cardinality at most m - 1 
leads to an O(n^), with k = max(m,r), solution algorithm for the m machine unit time scheduling 
problem (cf. the choice verification algorithm) and to an 0(n^ ) with k'= max(m + l,r) solution 
algorithm for the m machine unit time scheduling problem with release and due dates. This also 
holds for m- profiles instead of m machines! 

Lemma 6 follows immediately from what was said above and has the nice feature that the 
consideration of release and due dates increases the exponent of the time bound by at most 1. If the 
time for the identification of test choices exceeds m, the additional consideration of release and due 
dates (and m-profiles) has no effect at all on the time bound. Lemma 6 might, for instance, be of 
great use in connection with special problem classes such as interval orders or forests, which 
are discussed next. For w.r.t. these two special classes, a polynomial time solution for the 
additional consideration of release and due dates is not yet known. However, the application hinges 
on the identification of an appropriate test choice finding mechanism, as the special mechanisms 
identified below for interval orders and forests without release and due dates do not apply to the 
extension in Lemma 6. So the requirement of a general test choice finding mechanism in Lemma 6 
is essential in order to obtain the result 
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We next consider the special class of so-called interval orders, i.e. posets (A,0) that have an 
interval order representation in the following sense: To each a e A an interval [u a , o a ] c[R + l 
can be assigned, such that for a * P we have (a, p) e 0 <=» o a < up. Interval orders are of great 
importance in scheduling (cf. [19], [24], [25]), as any schedule naturally induces an associated 
interval order. There are a number of equivalent characterizations of interval orders [5], [1 1], [12], 
[25]. We mention here the characterization via the forbidden 4-element subposet given in Figure 4 
and the following criterion: 

(A,0) with |A| = n is an interval order iff there is a ordering ccj, a 2 ,.-,oc n among the jobs such that 
N((Xj) c N(ocj) holds whenever i < j. I.e. (A,0) is an interval order iff the set of successor-sets of 
jobs is totally ordered w.r.t. set inclusion. (This holds analogously for the sets of predecessors of 
jobs). Whether (A,0) is an interval order can be tested in O(n^), and if it is, an associated ordering 
of the jobs can also be given in that time. In the following, we assume jobs to be already numbered 
accordingly. 




Fig. 4: Diagram of the forbidden subposet whose 
absence characterizes interval orders 



LEMMA 7: (cf. also [22]) 

m-machine unit time scheduling of interval orders is possible in O(n^) time for any m. 



Proof: 

Note first, that the normalization to |A| = m*t by adding independent elements remains in the class 
of interval orders. As mentioned above, we can then assume within the time bound O(n^) that jobs 
are ordered w.r.t. decreasing sets of successors. Now given any considered set B with 1 < |B| < m, 
we pick as a choice a job ae A for which N(a) is maximal among all jobs in B. Obviously, this 
yields a test choice (which is obviously non-blocking). Even more a being a choice for some 
set B with |B| > 2 does not have any influence on the bound for a beyond the influence resulting 

from B = {a} anyway, as N(a) = kj N(y) will be true. Therefore, the computation can be 

yeB 

restricted to sets B with |B| = 1, i.e. steps 2 to 4 in the choice verification algorithm for sets B with 
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|B| = 1 are sufficient to solve the problem. I.e. in O(n^) time, a decision is obtained, whether there 
exists a valid schedule and if it does, a schedule is determined in that time. Of course it is also 
immediately obtained, that priority scheduling w.r.t. the mentioned ordering of the jobs yields 
exactly the same solution as the one arising with the choice verification algorithm, as described 
above. In fact, that is the solution proposed in [22], which has the same time complexity. • 



The last special class of posets treated in this paper are forests. Forests are a natural generali- 
sation of trees, which is preferred here for the following reason: while an application of the 
normalization procedure (adding independent jobs to a poset) leaves the class of trees, it stays 
within the class of forests. On the other hand, the known algorithm for trees [13], [14], [27] 
applies to forests as well. Concerning the treatment of forests, one distinguishes inforests and 
outforests. Inforests and outforests can be transformed into each other by a simple dualization. 
This holds similar for associated schedules. This allows to restrict the algorithmic treatment 
essentially to one of the two cases. 



LEMMA 8: (cf. also [13], [14], [27]) 

m-machine scheduling of forests is possible in O(n^) time for any m. 



Proof: 

As mentioned above, one can w.l.o.g. restrict attention to the case of outforests. The classical 
approach in [14] then consists in determining in O(n^) time complexity the dual height of all jobs 
and then doing priority scheduling w.r.t. the priority ordering among the jobs, induced by 
increasing dual height of jobs; i.e. jobs of greater dual height are preferred. The correctness of this 
approach follows from a quite straightforward inductive argument. In our context, the argument is 
(in an intuitive description) as follows. W.l.o.g., one can start with m jobs. Let S denote a set of m 
minimal elements with lexicographically greatest dual height. Assume that starting with S does not 
lead to a valid schedule, while starting a set S' of minimal elements which are not lexicographically 
minimal does. In this case, we apply the inductive hypothesis to the subproblem arising after 
starting S'. This leads to a schedule for the subproblem that arises from priority scheduling w.r.t. 
the dual height. It is not difficult to see that within this schedule, successors of elements of "low" 
dual height in S' will w.l.o.g. not appear until all elements from S are scheduled. Here the main 
observation is that for outtrees, after finishing a job from S, a successor of this job becomes 
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available, whose dual height is still greater on equal to the dual height of those jobs with "low" dual 
height in S'. Seeing this, we can exchange each job from S, scheduled later, with a job of "low" 
dual height, scheduled in the first time slot. This yields a valid schedule, starting with S, a 
contradiction to our assumption. 

Of course this result solves the problem in O(n^) time via priority scheduling w.r.t. the 
dual height. It is illustrative to observe, however, that this result also yields a (non-blocking) test 
choice by chosing within a set B of cardinaliy 1 < |B| < m a job with greatest dual height. The proof 
of this statement is as follows: If the problem has no solution, every choice will lead to a bound 
contradiction for the associated bounds, due to the Representation Theorem. If there is a solution, 
priority scheduling w.r.t. the dual height will yield a valid schedule, due to the argument given 
above. W.l.o.g. we can assume that w.r.t. this schedule, the chosen element for a set B with 1 < 
|B| < m (which has maximal dual height in B) will be scheduled earliest. As argued in connection 
with the Representation Theorem this is enough to prove that the schedule is compatible with the 
choice-induced bounds. This in turn guarantees that no bound contradiction will occur. Thus the 
test-choice property is established. 

It is interesting to note that the determination of the test-choice depended bound may, in case 
there is a solution, lead to a different kind of priority schedule then priority scheduling w.r.t. the 
dual height, cf. Example 9 for this phenomenon. I.e. in the sense of the Representation Theorem, 
priority scheduling according to the dual height may not necessarily lead to the lexicographically 
smallest start set w.r.t. the test-choice induced bound. However, both ways of scheduling lead 
either to valid schedules or establish the non existence of a solution, a potential coincidence that is 
included in the formulation of the Representation Theorem. Needless to say that, from a complexity 
point of view, O(n^) priority scheduling w.r.t. the dual height is much superior then the 0(n m ) 
priority scheduling w.r.t. the choice-induced bounds. ■ 



EXAMPLE 9: 

Consider the 3-machine unit time scheduling problem [(A,0), t] with A = {1,...,18}, 0 an out- 
forest as given in Figure 5 by a diagram and t = 6. 
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A priority list on the non-maximal elements of (A,0) compatible with the dual height of jobs, is 
1<4<7<10<15<2<5<8. A corresponding feasible schedule for the problem is given by a 
Gantt- chart in Figure 6a. 




Next we consider the (non-blocking) choice on sets of two elements, induced by the dual 
height of jobs in the way stated above. As an application of the choice verification algorithm, we 
obtain the corresponding bounds for all non-maximal elements as b(15) = 4, b(l) = b(4) = b(7) = 4 
and, finally, b(10) = 3. A corresponding feasible schedule for the problem is given by a Gantt-chart 
in Figure 6b. Note that this schedule is not compatible with priority scheduling w.r.t. the dual 
height of jobs, which requires the set { 1, 4, 7} of jobs as start set. 
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Notice, finally, that the schedule given in Figure 6a, also fulfills the statement of the 
Representation Theorem in the sense that after removing jobs {1,4,7}, there is no bound 
contradiction on the remaining jobs w.r.t. the bounds t - 1, and so forth. 

To close this Section, we mention that there are some other nice special cases solved like 
opposing forests for m arbitrary, cf. [4], posets of bounded height r for arbitrary fixed m 
with 0(n r ( m " 1 ) +1 ) time complexity, cf. [3], and, finally, posets of bounded width r for m 
arbitrary with 0(n^ r ) time complexity, cf. [1], [23]. This last mentioned algorithm consists in 
dynamic programming w.r.t. the so-called fronts (i.e. minimal sets of not yet scheduled jobs) 
that may be encountered during on-line scheduling. The number of such fronts is bounded by n r 
for posets of bounded width r. Note that membership in all three special classes can be verified 
efficiently. (For another reference to posets of bounded width see Theorem 10 (3) below). 

We mention finally, that up to now no definite insight (as discussed in this Section for some 
other special cases) is available for possible connections between these special cases and the 
corresponding choice-test problem in the sense of the Representation Theorem. The same is partly 
true w.r.t. the possible incorporation of release and due dates and, furthermore, m-profiles. How- 
ever, there is presently work going on concerning these questions at the University of Passau. 



4. TOWARDS A POLYNOMIAL TIME ALGORITHM FOR GENERAL 
M-MACHINE UNIT TIME SCHEDULING PROBLEMS 

A major insight from the Representation Theorem and the choice verification algorithm is, that 
somehow a solution to the choice problem for sets 1 < |B| < m is the main conceptual - and 
probably also algorithmic-step to make in order to obtain a general algorithm. However, while we 
saw in Section 3 that this may be done quite nicely in special cases, there is, unfortunately, not 
much insight available up to now for this problem in the general case. 
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Therefore, we will take a slightly different route in this Section (where we will omit the 
normalization condition |A| = ml). We will try to accumulate (in polynomial time) information 
concerning the placing of all jobs. The available information will be collected in the form of a 
bounding system, that gives lower and upper bounds L(a) and U(a) for all elements a € A. 
In this bounding system, we will particularly incorporate bounds for sets B with 1 < |B| < m in 
those cases, where at most one element from B can accept the corresponding lower or upper 
bound L(B) or U(B) for B without causing a bound contradiction (so-called forced choices, cf. 
the validation step described below). In order to get an intuitive idea of this last point, compare 
Example 1 (2). Here, at a certain point, we found - using the bounds for single elements - the set 
bound b*({l,8}) = 1. As job 8 has predecessors and therefore cannot be finished by time 1, we 
could deduce the bound 1 for job 1. I.e. here, the choice on the set {1,8} is forced. Actually, 
later on a contradiction was obtained in this example, using this observation, i.e. there is no 
solution at all in the example. 

When talking about a bounding system, we will require it to be valid and monotone in the 
following sense: If there are feasible schedules T for the problem, then for all a € A and all such 
T, L(a) < T(a) < T(a) + 1 < U(a) must be true. The same holds analogously for bounds of sets of 
higher cardinality. Call a bounding system monotone, if adding new relations or further elements 
or decreasing t will not decrease lower and not increase upper bounds; again also w.r.t. sets instead 
of elements. Call a bounding system not contradictory if 0 ^ L(a) < U(a) < t holds for all a e 
A and if for all 0 < i < j < t, no bound contradiction of the form |{a e A | i < L(a) < U(a) < j} | > 
m(j - i) occurs. Obviously, given a valid bounding system, this has to be not contradictory in order 
to allow the existence of a valid schedule for the problem. Our hope is that the contrary is also true, 
i.e. as a kind of WORKING-CONJECTURE, we hope for the existence of a bounding system, 
deducable in polynomial time, such that the bounding system not being contradictory guarantees the 
existence of a feasible schedule for a given problem. 



THEOREM 10: 

If the working-conjecture is correct for a bounding system (for the m-machine case), and if the 
bounding system is computable in O(n^) time (k > 2), then the general m-machine unit time 
scheduling problem can also be solved in polynomial time. In particular, the following time bounds 
can be established: 

(1) The existence of a solution for the problem can be decided in O(n^) time: 

(2) If a solution exists, a feasible schedule can be obtained in 0(n^ + ^) time. 
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(3) For posets of bounded width, the feasible schedule in (2) can even be constructed in 
(Xn^+l) time. 



Proof: 

(1) The algorithm consists in the repetition of a basic step, which essentially consists in the 
computation of the bounding system for individual jobs for [(A,0),t] in O(n^) time. In O(n^) time it 
is then checked, whether this system is contradictory. If it is, the problem has no solution, 
otherwise it has, due to the Working Conjecture. This solves (1). 

(2) Assume that there is a solution for [(A,0),t]. As we will demonstrate below in Examples 12 to 
16, the bounds for jobs obtained from the computation of the bounding system in (1) may be 
informative w.r.t. finding a valid schedule. However, we will also demonstrate in Example 14 that, 
in general, additional computational effort is needed. A straightforward way to proceed is to add a 
further relation (a, p) with a * p and a 0-independent from p to (A, 0) and check with the basic 
algorithm in (1), whether the resulting problem [(A, 0"), t] is solvable. If it is not, we substitue 
(a, P) by a different relation. Otherwise, we proceed from [(A, O'), t] as described for (A, 0). 
After at most O(n^) steps, we arrive at a problem [(A, 0*), t], which has a solution, but for which 
any proper order extension of 0* does no longer allow a solution. It is easily checked that each 
such poset 0* consists of a linear sequence of at most t blocks of at most m independent elements 
each. This sequence corresponds bijectively to a feasible schedule, that is thus obtained in 0(n^ + ^) 
time. 

(3) Another way to produce a valid schedule in (2) is due to the following consideration. As the 
problem has a solution, certain minimal elements a of (A, 0) can start at time zero. In the 
non-trivial case, there are more minimal elements available as machines. So, the question arises, 
which jobs to start. It is not difficult to prove, that the possibility to start a job a is equivalent to the 
statement, that the partial order (A, O') with 0' = {0 u {a, P) | p not minimal in (A, 0)} has a 
solution. So we could consider some minimal element a and check the existence of a solution for 
the associated poset (A, O') in O(n^) time. If the resulting problem has no solution, we try the next 
minimal element in (A, 0). Otherwise, we proceed from (A, O') with the next minimal element in 
(A, O') (which are the same minimal elements as those in (A, 0)). By proceeding through the 
minimal elements, we will at some point have identified m minimal elements which iteratively lead 
to solvable problems. It is immediate that these m elements can be scheduled at time zero and allow 
a continuation to a feasible schedule by applying the same procedure to the resulting subproblems. 
In proceeding this way, a 0(n^ + ^) procedure has to be applied at most n-t / m times. This yields 
again the 0(n^ + ^) time bound, also obtained in (2). However, for posets of bounded width r e 
IN, the number of minimal jobs to be considered in each time slot is bounded by the constant r. 
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(Note, that the basic poset extension operation, discussed in (2), does not increase the width!) 
Therefore, the time complexity for finding a valid schedule can be reduced to 0(n^ + *), as stated in 
the theorem. ■ 



In the following, we introduce a bounding system that can be determined in O(n^) time and 
that allows a number of insights in a variety of illustrative examples, some of which are treated 
below and in Section 5 of this paper. The given boundary system seems to be the natural core to 
start with for m-machine scheduling. It is certainly not sufficient for general me IN, cf. Example 
16 below. All studied examples so far indicate however, that the Working-Conjecture might apply 
to this bounding system for the 3-machine case. A report on some progress concerning a 
correctness proof for this statement is given in Section 5. On the other hand, we give in Example 
14 an indication on to how the bounding system might be tightened (still in polynomial time), if the 
given bounding system should turn out to be insufficient even in the 3-machine case. 



BOUNDING SYSTEM 

The system gives valid and monotone lower and upper bounds L(a) and U(a) on the starting 
and completion times of jobs (and sets of jobs) in O(n^) time. For a set B of jobs, a lower (upper) 
bound L(B) (U(B)) means that at least one job in B has to start not earlier than L(B) (has to be 
completed by U(B)). In addition, bounds are checked for consistency. 



(1) The basic step and the associated closure operator 

Let an m-machine unit time scheduling problem [(A, 0), t] be given, together with present lower 
bounds 0 < L(a) < U(a) < t for all a e A, starting originally form L(a) = 0 and U(a) = t for 
all a e A. We assume that the present bounds are consistent in the sense that |{as A | i < L(a) 
and U(a) < j} | < m-(j - i) for all pairs i, j with 0 < i < j < t. Then we update U(a) by exactly the 
basic formula used in Sections 2 and 3 w.r.t. the bounds b(a). (This holds similarly again for sets 
B with more than one element, where we now however consider only one such set at a time w.r.t. 
the bounds b(a) obtained, as described, for single elements a). The bounds U(B) and L(B) for |B| 
> 1 are defined analogously, i.e. via dualization. We first determine U(a) by backward recursion 
and afterwards L(a) by forward recursion for all a e A. Both definitions do not infere. In O(n^) 
time, all these values are obtained. In the same time complexity, it can be checked whether U(a) < 
L(p) holds for pairs a, P of 0-independent elements. If this is the case, we add all the 
correspondent relations (a, P) to (A, 0) to obtain an extension (A, O'). Obviously, any solution for 
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[(A, 0), t] must be a solution for [(A, O'), t], too. So as part of a closure operation, we can 
proceed with (A, O') and determine the corresponded bounds as part of the described basic step. 
Note that a basic step needs O(n^) time and has to be repeated at most n^ times. 



(2) The validation step and the associated closure operator 

If we stop with step (1), the question arises whether a job a with bounds L(a) and U(a) may 
really start at time L(a) or complete at time U(a) without causing a contradiction w.r.t. the bounds 
imposed on the other jobs. We can check this local validity of the obtained bounds by first 
fixing the placing of job a into the time slot [L(a), L(a) + 1) by putting (tentatively) U'(a): = L(a) 
+ 1 and afterwards by fixing the placing of job a into the time slot [U(a) - 1, U(a)) in an 
analogous way. As we will demonstrate below, this kind of fixing may lead to a bound 
inconsistency, because (e.g. w.r.t. the upper bound U(a)) for certain time windows [i, j] with 
L(a) < i < U(a) < j, a set A^ j of jobs that are not successors of a, must be placed within this time 
window, for which the number |Ay| exceeds m(j - i) - 1 {ye N(a) | b(y) < j}| - 1; in this context, 
on can w.l.o.g. restrict attention to the case where Ajj consists only of such jobs, that are 
independent of a. In this described case, U(a) must be reduced at least to i, which in particular 
means that the order relations (a, p) with Pe A^ arbitrary, might be added to the present partial 
order to obtain an extension similar to (1). In this case, one can as part of a closure operation 
again proceed with (1), afterwards again with (2) and so forth. Note that a validation step needs 
O(n^) time, as it actually means the performance of up to 2n basic bound computations and 
consistency checks. The complete closure operator resulting from (1) and (2) consists thus in 
at most n^ applications of either an O(n^) or O(n^) procedure, i.e. can be done in O(n^) time 
complexity. I finally stops in a "closed situation" in the sense that the bounds obtained so far do 
not lead to a further tightening of the bounds. 

A major aspect of the validation step is the incorporation of forced choices for sets B with 
1 < |B| < m in O(n^) time, i.e. without scanning all sets B of cardinality at most m - 1. This 
independence of m, which is demonstrated in the next lemma, is nice from a computational point of 
view. However, given the NP-completeness of m-machine unit time scheduling for m arbitrary 
[18], [28], is means of course the insufficiency of the given bounding system in connection with 
the Working-Conjecture w.r.t. general m e N. The case of m = 3 might, of course, be different. 
More comments on that follow below. 
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LEMMA 11: 

Consider an m-machine unit time scheduling problem [(A, 0), t] for which the associated bounding 
system is not contradictory. Then for every set B with 1 < |B| < m, there are at least two elements 
ocj, (X 2 e B and at least two elements Pj, P 2 e B fulfilling L(oq) < U(B) - 1 and U(pp > L(B) + 

1 for i = 1,2. 1.e. all forced or impossible choices for considered sets B (i.e. situations, where at 
most one element a, P from B fulfills L(a) < U(B) - 1 or U(P) > L(B) + 1) are already 
incorporated (via U(a) < U(B) or L(P) > L(B)) into the bounding system, due to the validation 
step. 



Proof: 



Assume that at some point in the computation of the bounding system, only one job a or p from a 
(currently) independent set B with 1 < |B| < m fulfills L(a) < U(B) - 1 or U(p) > L(B) + 1. We 
concentrate w.l.o.g. on the first case. Put i: = min (L(y) | ye B \ {a}} > U(B), chose j as a 
defining time point for U(B), i.e. such that U(B) = j - f | {8 e N(y) | b(a) < j} | / ml with j > 



Now consider the validation step for U(a) and the time window [i, j], where, because of 
the basic algorithmic step and the given independence of B, U(a) > i + 1 > U(B) + 1 holds. Here, j 
> U(a) can also w.l.o.g. be assumed, as otherwise a bound contradiction w.r.t. the set B \ {a} 
would be obtained. The validation step then yields U(a) < i and leads to the introduction of all 
relations (a, y) with ye B \ {a} to the poset, followed by the establishing of U(a) < U(B), i.e. to 
the forced choice, as stated. This solves forced choice situations. Obviously, a modification of this 
argument works for impossible choices, i.e. yields a bound contradiction, should no job from a set 
B be able to accept the corresponding bounds L(B) and U(B). m 



In the following, we apply the bounding system approach to a number of interesting 
3-machine unit time scheduling problems. Note that in connection with Theorem 10, which uses an 
appropriate introduction of new relations, we are using a combined O(n^) algorithm, for 
which there is some evidence that it might actually solve the 3 -machine case completely. 
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EXAMPLE 12: 

Consider the following 3-machine unit time scheduling problem [(A, 0), 5], where (A, 0) is given 
by a diagram in Figure 7. 




Fig. 7: A poset with unique starting set in an optimal 3-machine unit time schedule 



The problem was chosen, because it has a solution in which, furthermore, the associated 
schedule is essentially unique. As the problem has a solution, no bound contradiction can occur. 
However we will see, that the bounds obtained directly lead to the solution. 

The basic step in the computation of the considered bounding system yields U(2) = U(3) = 
U(4) = U(5) = U(6) = 2, U(8) = 3 and L(9) = 2. As a consequence, we can add the additional 
relations (4, 9), (5, 9) and (6, 9) to the poset. These leads to the modification L(9) = 3, yielding the 
introduction of the additional relation (8, 9). As a consequence, we obtain U(8) = 2 and thus U(4) 
= U(5) = U(6) = 1. The first two time slots have therefore to be filled by jobs {4,5,6} and 
{2,3,4}, respectively. As part of the validation step, L(l) = 2, U(l) = 3, L(7) = 3, U(7) = 4, L(9) 
= 4, U(9) = 5 are obtained, together with L(i) = 2 and U(i) = 5 for all jobs i = 10,..., 15. This 
determines the solution to the problem, which is unique up to permutations among the jobs 
10,..., 15. Note that the bounds obtained in this example exactly give the range, in which the 
respective jobs might be placed in valid schedules for the problem. 
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EXAMPLE 13: 

Consider the following (3-critical) unit time scheduling problem [(A, 0), 4] with |A| = 12 and (A,0) 
as given in Figure 8 by a diagram. This problem has a solution in 5 time units, however not in 4 
time units. 




The basic step in the computation of the considered bounding system for t = 5 yields U(l) = 
U(4) = U(5) = U(6) = 2 and L(3) = L(10) = L(ll) = L(12) = 2. As a consequence, we can 
introduce the additional relations (1,10), (1,11), (1,12) and (4,3), (5,3) and (6,3). Annother 
application of the basic step then yields U(4) = U(5) = U(6) = 1 as well as L(10) = L(1 1) = L(12) 
= 3. As part of a validation step, L(l) = 1 and consequently L(3) = 3 will be obtained. This yields 
the bound contradiction |{ae A | L(a) > 3} | > 3, i.e. there is no solution in 4 time units. 

EXAMPLE 14: 

Consider the following 3 machine unit time scheduling problem [(A, 0), t], where (A, 0) is given 
by a diagram in Figure 9. 

(1) The basic step in the computation of the considered bounding system yields U(10) = U(1 1) = 
5, U(8) = U(9) = 4 and U(i) = 3 for i = 1,...,6. It is easily seen that all these bounds are really 
obtained in valid schedules, so they cannot be improved. In particular, the validation step and the 
consideration of jobs B with |B| = 2 as part of the computation of the bounding system will not 
further decrease these bounds. 
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(2) It is interesting to note, however, that because of L(10) = L(1 1) = 2, the consideration of the 
set B = {10,11} with associated bound U(B) = 3 (which can be fulfilled by both jobs and thus 
does not lead to the introduction of new relations) implies that either jobs {1,2,3} or jobs {4,5,6} 
have to be chosen as starting set. This cannot immediately be deduced from the considered 
bounding system, i.e. additional steps as described in Steps 2 and 3 in Theorem 10 are needed 
here, cf. also (4) below. Note, however, that it could be deduced here that necessarily L(7) > 1 
holds. This sharper bound could have even been obtained within the same time complexity. I.e. we 
see, that the bounds resulting from the considered bounding system are usually not sharp. Also 
we see, how they might be tightened if the Working-Conjecture does not hold w.r.t. the 
considered bounding system even in the case of m = 3. 

(3) Along the lines followed in (2), it can be shown that jobs 8 and 9 will either be scheduled at 
time 2 or at time 4 or 5 in a feasible schedule for the problem. I.e. the possible time points, where 
jobs may be placed in feasible schedules, are, in general, no intervals. 




Fig. 9: A poset with an optimal 3-machine unit time schedule length of 7 time units 



(4) If we apply the procedure described in connection with Theorem 8 in order to find a feasible 
schedule for the problem, we might first try to start job 7 at time 0 (which is impossible) by adding 
the additional order relations (7, i) with i = 8,. ..,21. A bound contradiction would then, however, 
be obtained as follows. Because of the additional relations, L(8) = L(9) = 2 and thus L(10) = L(1 1) 
= 3 holds. Obviously, this contradicts U({10,1 1}) = 3, which would lead to a bound contradiction 
via the validation step. 

(5) If, instead, we try in (4) to start job 1 at time 0 (which is possible) by adding the additional 
relations (1, i) with i = 8 ,..., 21 and would afterwards continue with the idea to start job 4 at time 
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0, too, (which is impossible then), we would obtain (after adding the additional order relations (4, 
i), i = 8 ,.. .,21) exactly the same bound contradiction as given in (4). 

EXAMPLE 15: 

Consider the following 3 machine unit time scheduling problem [(A, 0), 26], where (A, 0) is given 
by a diagram in Figure 10a. Note that the problem, that consists in the treatment of two parallel 
chains of independent elements is quite involved; cf. also Section 5 for this kind of structure. As 
the problem has a solution, no bounding contradictions will occur. The example was obtained as a 
counterexample for inductive proofs for certain (false) conjectures in the field. The main point is 
that the 12 independent minimal jobs 43,. ..,54 have to be scheduled within the first 4 time slots. 
This can be obtained from the considered boundary system as follows: 




Fig. 10a: A poset (A, 0) with |A| = 78, that allows unit time scheduling on 3 machines in 26 
time units. In any optimal schedule, jobs 43,..., 54 have to be scheduled within the 

first 4 time slots. 



(1) We first obtain L(21) = 10 and U(43) = ... = U(54) = 10. This will lead to the introduction of 
the new ordering relations (i, 21) with i = 43,.. .,54 and the modified lower bound L(21) =11. 

(2) Given (1), i.e. L(a) > 12 for all successors of job 21, a basic step w.r.t. the originally 
obtained bound U(56) = 12 leads to the modification U(56) =11, implying U(55) = 10. 
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(3) Given (2), a validation step for the originally obtained bound L(17) = 8 yields, that because 
jobs 55 and 17 (or 18) together have 28 predecessors and job 56 has to be finished by time 10, the 
new bounds L(17) = L(18) = 10 are obtained. This immediately implies L(21) =12. 

(4) A validation step for U(59) = 15, which was originally obtained, leads because of L(21) = 
12 and a total number of successors of jobs 59 and 21 of 40 to the updated upper bound U(59) = 

12, implying U(55) = 8. 

(5) A validation step for the original bound L(1 1) = L(12) = 5 now yields, given that U(55) = 8, 
the adapted values L(ll) = L(12) = 8. This directly implies L(21) = 13. 

(6) A validation step for the original bound U(62) = 18, given that L(21) = 13, yields U(62) = 

13. This implies U(56) = 7. 

(7) A validation step for the original bound L(7) = L(8) = 3, given that U(56) = 7, yields L(7) = 
L(8) = 7 and therefore leads to the introduction of the new relations (56, 7) and (56, 8). This 
implies U(56) = 6 and consequently U(i) = 4 for i = 43,..., 54. This essentially solves the problem. 

A complete determination of the associated bounding system gives sharp bounds for the 
placing of all jobs. A solution, which is essentially unique, is given subsequently by the sequence 
of sets of 3 jobs for all time slots 1,...,26: 
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Fig. 10b: A feasible schedule for Example 15. 



Examples 12 to 15 have shown the potential of the used bounding system for a number of 
3-machine unit time scheduling problems (for further examples cf. Section 5). As already 
mentioned, there is some hope that this procedure with O(n^) time complexity really solves the 
3-machine case (or at least may be extended towards a sufficient bounding system along the lines 
described in connection with Example 14). Approaches towards a proof of this statement will 
follow in Section 5. Before, we will indicate, where problems with this bounding system may arise 
for m = 3 and have to arise for increasing values m e M. As we will demonstrate in the 
concluding example of this Section, the problems are of the following type: It may well be that, for 
all sets B of jobs with 1 < |B| < m, at least two jobs from each set B may be able to accept the 
bounds L(B) and/or U(B). The question still is, whether all these requirements can be met 
simultaneously. I.e. the question is, whether a particular start set can satisfy all requirements of 
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typ U(B) = 1 for all sets B considered, and if it can, whether a set of jobs for the second time slot 
can be found, that together with the starting set fulfills all requirements of the form U(B) = 2 and so 
forth. Incorporating checks of the type described into the determination of a bounding system might 
be the right way to extend the approach followed in Section 4 to greater values of m. Here, the need 
to check sets of m minimal elements concerning the described behaviour would cause the increase 
in time complexity to an at least linear dependence on m, that has to be expected. 



EXAMPLE 16: 

The example is essentially obtained from the reduction of CLIQUE to the m-machine unit time 
scheduling problem with m arbitrary and t = 3 that is given in [18]. We start here from the graph G, 
given in Figure 11a and transform it into the unit time scheduling problem [(A, 0), 3], with m = 7 
and (A, 0) as given by a diagram in Figure lib. The graph has no clique of cardinality 3 which will 
mean, that the scheduling problem has no solution. 




Fig. 1 la: A graph G without cliques of 3 elements (so-called K 3 3 ) 



Note that any job from { 1 ,..., 6 } (which correspond to nodes of G) has exactly 3 successors, 
while any job from 7,..., 15 (which correspond to vertices of G) has 2 predecessors, viz. the nodes 
that are incident with the associated vertex. For reasons of symmetry it is easy to see, that the 
associated bounding system will correctly place jobs 16,. ..,21 and will lead to the bounds L(a) = 0 
and U(a) = 2 for a = 1,...,6 and L(a) = 1 and U(a) = 3 for a = 7,..., 15. It is then easily 
obtained that particularly interesting 0-independent sets B in the sense of the Representation 
Theorem and Corollary 2 are as follows: they contain job 20, furthermore a set C of three jobs from 
{1,...,6} and finally, at most 2 further jobs from {7,..., 15}. Here, it is easily obtained from the 
construction, that the latter jobs are determined by C in the sense that, given any set B as 
considered, there are at least 7 successors in (7,..., 15}, due to the fact, that G contains no cliques, 
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i.e. 3 edges of G are incident with at most 2 vertices from G. Obviously, this implies the bound 
U(B) = 1 for all such sets B. Though in all cases, every element in B (with |B| = 3) can 
individually accept the upper time bound 1 because of L(a) = 0 for all a e {1,...,6}, this does 
not mean, that all these requirements can be satisfied simultaneously. To the contrary, whatever 
3-elementary set of jobs from {1,...,6} is started at time 0, there remains a subset B with |B| = 3, 
from which no element was chosen. This shows the non-existence of a solution for the 
considered 7-machine unit time scheduling problem. [Note that similar examples have up to now 
not been found for the 3-machine case. Therefore, the O(n^) procedure resulting from the given 
bounding system and Theorem 10 might still be sufficient to solve this case]. 




Fig. lib: The poset (A, 0), essentially obtained from G via a standard reduction form CLIQUE. 



5. M-CRITICAL POSETS 

In Section 4, the question was raised of how to prove the intended result that a certain bounding 
system solves an m-machine unit time scheduling problem in the sense that if no bound 
contradiction is obtained, there is a valid schedule. Certainly, this question arises particularly in 
connection with the proposed bounding system for the 3-machine case. Such proof can be expected 
to be laborious, given the involved recursive nature (as closure operator) of the proposed bounding 
system already for the case m = 3. 
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In the following we try to follow an approach, that was already successful in other 
combinatorical optimization problems, viz. the concentration on special critical problem instances. 
The hope is that such critical instances may have a special structure and that this structure can be 
exploited.In addition, their efficient characterization would yield a proof of membership of the 
considered problem in CO-NP. 



DEFINITION: 

Call a poset 0 = (A, 0) m-critical, if an optimal unit time schedule for this problem on m 
machines needs t time units, while after removing any element from 0 (or A), an optimal schedule 
needs only t - 1 time units. 

Examples of 3-critical posets were already considered, cf. Examples 1 and 13. We show next, 
that for a proof concerning the correctness of a valid and monoton bounding system for m-machine 
unit time scheduling, only the instances [(A, 0), t - 1], where (A, 0) is m-critical and t is the optimal 
schedule length for (A, 0), need to be considered. 



THEOREM 17: 

Assume for a valid and monotone bounding system for m machine unit time scheduling, that it 
yields a bound contradiction for all problems instances [(A, 0), t - 1], where (A, 0) is m-critical 
with optimal schedule length t. Then the bounding system is correct, i.e. it yields a bound 
contradiction for some arbitrary problem instance iff there is no solution to the problem. 



Proof: 

As the bounding system is feasible, any bound contradiction implies that the respective problem 
instance has no solution. 

Assume, for the other direction, that there is no bound contradiction for a problem instance 
[(A,0), t]. Then one has to show that there is a solution to the problem. We prove this by induction 
on 0, regarded as a set of pairs. The case of a trivial partial order (only independent elements) 
follows immediately. The induction step is then as follows. We assume that [(A, 0), t] has no 
solution. W.o.l.g. we can assume that [(A, 0, t + 1] has a solution, for otherwise, due to the 
monotonicity behaviour of the bounding system, there would be certainly no bounding 
contradiction w.r.t. t + 1 and t could thus be replaced by t + 1 in the argument. By the 
induction hypothesis, we know the correctness of the bounding system after eleminating any 
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relation or any element from (A, 0), leading to a poset (A', O') with A' c A and 0' c0. Again, 
relying on the monotonicity of the bounding system, there will certainly be no bound contradiction 
resulting from an application of the bounding system to [(A', O'), t]. Because of the induction 
hypothesis, this implies that (A', O') can indeed be scheduled in t time units. Thus we have 
obtained a problem instance [(A, 0), t], for which there is a solution in t + 1 but not in t time units 
and for which any relaxation of (A, 0) allows a solution in t time units. I.e. we have obtained an 
m-critical poset (A, 0), for which the bounding system produces a bound contradiction by 
assumption. This is a contradiction to the behaviour required in the theorem, concluding the proof. 



Because of Theorem 17, the consideration of m-critical posets is sufficient in proofs of the 
correctness of valid and monotone bounding systems as considered in Section 4. In a certain sense 
(cf. e.g. the proof of Theorem 17) they constitute the most difficult problem instances to deal with. 
Still, there is some hope that such "extreme" problem instances might have such a special structure, 
that this may really facilitated a proof. This idea was e.g. carried out in [26] and was, at least 
partially, successful. The major step in this direction was the subsequently formulated Theorem 19, 
that characterizes the 2-critical posets exactly as the class of so-called odd weak orders, 
where 0 = (A, 0) is called an odd semi-order, if 0 consists of a chain of autonomous subposets, 
each of which consists of an uneven number of independent elements, cf. the example, given next. 
The proof of the statement of Theorem 19 was done in [26] via an application of Hall's marriage 
theorem. Here, we will present the outline of an alternative proof that is more direct. 



EXAMPLE 18: 

Figure 12 shows an odd semiorder 0 = (A, 0) of 19 elements by a diagram. The associated 
minimal schedule length on 2 machines is t = 14. 




Fig. 12: An odd weak order 
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THEOREM 19 (cf. [26]) 

The 2-critical posets are exactly the odd weak orders. 

This theorem is e.g. also implicitly contained in the correctness proof of certain algorithms for the 

2-machine problem [2], [7]. They consider partitions of A into pairwise disjoint sets 

and U such that (a, (3) e 0 whenever a e Aj, p e Aj with i < j. In other words, the subposets 

GIAj,... 0|A m are series -connected. Obviously, any such partition gives a lower bound 
m 

iriAii/i] on the optimal schedule length. It is shown in these papers that there exists a partition 
i=l 

such that the associated lower bound is in fact equal to the length of the constructed schedule, 
thereby obtaining a correctness proof and an (implicit) Min-Max-Theorem. 

Deleting from such an optimal partition the set U, all order relations among elements from the 
same Aj, and one element from each even Aj then gives an odd weak order in the sense of Theorem 
19. 



The algorithm in [7] constructs such an optimal partition while producing the linear extension. 
Basically the idea is to have all elements in 0 on their lowest possible level, and to traverse the 
order from the top level by level and construct an optimal schedule. If a level is even, it presents no 
problem, if it is odd, an element from the closest lower level is pulled up according to a choice rule 
motivated by the "greedy" principles (make a choice that interferes as little as possible with later 
pull-up options). The pull-up operation is used to define the partition into the Aj (a pull-up occurs if 
there is (almost) a series connection between adjacent levels). 

All these techniques were used with the aim to prove optimality of the algorithm, and not to 
give a characterization of the 2-critical posets. We present here a simple proof of the 
characterization without aiming at their algorithmic construction. 



Proof of Theorem 19: 

It is easily seen that an odd weak order is 2-critical. 

In the other direction, we start from a given 2-critical poset 0 = (A, 0). It is easily verified by an 
inductive argument that 0 is an odd weak order, if 0 can properly by decomposed linearly into 
(autonomous) subposets [26]. If we assume that 0 cannot be decomposed this way and, further- 
more, know that an optimal unit time schedule for 0 on 2 machines needs not more than [ |A| / 2l 
time units, than obviously 0 consists of |A| independent jobs and |A| is uneven, i.e. 0 is again an 
odd weak order. We are therefore done, if we can show that these are all cases occuring. I.e. we 
will start now from a poset 0 = (A, 0), for which an optimal unit time schedule on 2 machines 
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needs t time units, where t > [ |A| / 2~1 (which implies, that 0 cannot consist of independent 
elements). We will show that in this case, 0 can properly be decomposed linearly into 
(autonomous) subpose ts. 

To begin with, we know already, that there are relations (a, P) e 0 with a * p. It is easily 
seen that we can chose a such that all successors of a are maximal elements in 0 and that the 
number of successors of a is minimal under the required property. As 0 is 2-critical, elimination of 
the relation (a,p) from (A, 0) leads to a poset, that can be scheduled in t - 1 time units on 2 
machines. Let T denote an associated schedule. Obviously, jobs a and p fill a time slot in this 
schedule. We can therefore transform T into an optimal schedule T' of the original problem by only 
changing T(P) to t - 1. It is straightforwardly seen, that all y fulfilling T(y) > T(a) are exactly the 
elements from N(a) and that the number of such elements is uneven. Also, the time slot [T'(oc), 
T'(oc) + 1) for T' can, w.l.o.g., be assumed to be the latest not completely filled time slot in any 
optimal unit time schedule for 0. 

We will now show that 0 properly decomposes linearly into the (autonomous) subposet 0* 
= (A*, 0*) with A* = A \ N(a) and the subposet 0 | N(a), consisting of an uneven number 2r + 1 
of independent jobs. Here, the length of an optimal unit time schedule for 0* is s and the equality 
s + r + 1 = t holds. What we essentially have to proof is that not only job a, but any job from A* 
is a predecessor of any job in N(a). Assume this not to be the case for a particular job p 0 , where 
w.l.o.g., p 0 can be assumed to be maximal in 0*. Certainly, if we could schedule 0* on 2 
machines optimally in a way, that only job P Q is scheduled in the last time slot [s, s - 1), a 
2-machine schedule for 0 with schedule length t - 1 could be obtained, a contradiction. 
Consequently, any such p Q will never occur in the final time slot. Also, eliminating such a job P 0 
from 0* would - for the resulting poset - not decrease the optimal schedule length s on 2 machines. 
This means, that because of induction, we can find within 0* a chain 0* | < 0* IL 2 ^ ^ 

0* |L^ of trivial posets 0j*:= 0 | Lj, j = l,...,k, such that 0j* consists of ij: = 2nj + 1 inde- 
pendent elements, j = l,...,k and such that X ( 11 ; + 1) = s holds. 

j=l 

By what was said it is easily obtained that 0* can be scheduled in such a way, that only the 

k-1 

2n^ + 1 independent elements from 0^ are scheduled in the time interval [ X (nj + 1), s). This 

j=l 

implies, that all elements in have to proceed all elements from N(a). Consequently, 0j* < ... 

< 0^* < 0 | N(a) forms an odd semiorder of associated minimal unit time schedule length 
k k 

X (nj+1) + (r + 1) = t, lying properly within 0, given that p 0 does not belong to the set LJ L:. 

j=i J j=i J 

This is a contradiction to the assumption of 0 being 2-critical, concluding the proof. ■ 
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As was already mentioned in [26], the characterization obtained in Theorem 19 yields another 
way to solve the 2-machine unit time scheduling problem, cf. Section 3. I.e. w.r.t. the simple 
bounding procedure, discussed in Section 2, one has only to verify, that there will be a bound 
contradiction, whenever an odd semiorder is considered for a schedule length t', which is too small 
to find a 2-machine schedule. But this is straightforwardly verified. Thus Theorems 17 and 19 
together mean, that the 2-machine bounding procedure indeed works correct for all posets. 

Some hints to m-critical posets for m > 2 are also given in [26]. Again, any such poset can 
be composed in a (may be trivial) linear sequence of m-critical posets. However, contrary to the 
2-machine case, a not further linearly decomposable m-critical poset 0 = (A, 0) may have an 
optimal m-machine unit time schedule length which properly exceeds T |A| /ml, cf. Examples 1 
and 13 in this paper. In [26], the hope was expressed that such posets may always be of the type 
occuring in Examples 1 and 13, i.e. may consist of at most m - 1 independent chains, consisting 
each of a non-trivial linear sequence of blocks of independent elements. Such posets seem to be 
very typical as nasty candidates for m-machine scheduling, as potentially there may be problems in 
filling all m time slots per time unit appropriately. Also, the hope was raised in [26] that for posets 
of this special type, the correctness of bounding systems as discussed in Section 4 might be 
proved. 

As concluding subject of this paper, we will add some more information to the points just 
mentioned. We will deal with the "easiest” case available, i.e. 3-machine scheduling of 3-critical 
posets 0 = (A, 0), that are not linearly decomposable, have at least 3 minimal and 3 maximal 
elements and that have an optimal unit time schedule length on 3 machines of t = [ |A| / 3 i + 1. 
This is exactly the class of posets, covering e.g. Examples 1 and 13 of this paper. We will 
particularly be concerned with the question of how to obtain new examples of this type by adding 
appropriate new maximal elements to an already found example. In doing so, we want to 
demonstrate simultaneously the modification of the bounding system approach used in Section 4 
for covering all those extensions. Furthermore, we want to obtain more insight into the nature of 
such 3-critical posets. Actually, we will find examples that are somewhat (but probably not 
decisively) more complicated as the type suggested above. 



EXAMPLE 20: 

Consider again the poset 0 = (A, 0), discussed in Example 1 and given by a diagram in Figure 1. 
This poset has 4-minimal and 7 maximal elements, is not linearly decomposable into autonomous 
subposets, is 3-critical and needs 5 time units for execution, while |A| = 12. Apparently, 0 falls 
into the class of problems considered here. As was already discussed in connection with Example 
1, optimal on-line scheduling of this poset will either lead to the bottleneck { 1,8} at time 2 or to the 
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bottleneck {i,8} with i e {5, 6, 7} at time 3. If we try to schedule the poset in 4 time units, the 
bounding system approach would lead to a contradiction. This can already be obtained from the fact 
that L(8) = 1, U ({1,8}) = 1 and thus U(l) = 1, and U({i,8}) = 2 for i e {5,6,7} and thus U(i) = 
2 for i e {5,6,7}. This implies L(8) > 2 and contradicts the bound U(2) < 2. 




If we now try to extend 0 by adding iteratively new maximal elements, then these elements 
should be successors of either jobs 1 and 8 and of either jobs i e {5,6,7} and 8. This is needed to 
allow a reduction of the schedule length, if a relation of type (l,i) with i e {5,6,7} or of type (8,j) 
with j e {9,..., 12} is eliminated. As similar reductions have iteratively to be guaranteed w.r.t. 
newly introduced elements, we arrive at extensions of the type given in Figure 13 by a diagram and 
varying parameters i, j e IN. 

Obviously, 0jj can optimally be scheduled on 3 machines in 5 + (i + j) time units while the 
number of jobs is 12 + 3 (i + j). A bound contradiction for the bounding system considered in 
Section 4 is obtained as follows. Again we have U ({1,8}) = 1 and L(8) = 1, implying U(l) = 1. 
Also, because of U({i,8}) = 2 for i e {5,6,7}, U(5) = U(6) = U(7) = 2 will be obtained. If we 
now proceed this way with the j newly introduced immediate successors of {5,6,7}, these will 
have to be finished by time 3 and so forth, leading to the completion of the i-th newly introduced 3 
jobs at time i + 2. This implies L(8) > i + 3, while also U(8) = i + 3 is obtained. This is a bound 
contradiction. 
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EXAMPLE 21: 

We consider the 3-critical poset 0 = (A, 0), given in Figure 14 a by its diagram. The poset is of the 
type, wanted, and was obtained as a modification of the poset given in Figure 1 and used in 
Example 20. The example leads to a class of even more involved 3-critical posets, as considered 
here. 




Fig. 14a: A 3-critical poset 



The only possible extension by 3 elements of 0 = (A, 0), that are maximal in the resulting 
poset 0* = (A*, 0*), which is 3-critical of the type considered, is given in Figure 14b by a 
diagram. 




Fig. 14b: A 3-critical poset, that is an extension of the 3-critical poset in Fig. 14a 
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(1) Note that here we obtained an example that consists no longer of two independent 
chains of linear sequences of independent blocks, as job 15 follows jobs 7 and 8. Still, the 
deviation from this expected type is not too far, given that job 15 is maximal. In fact, another 
3-critical poset with this phenomena is obtained from 0 = (A, 0) in Figure 9 in Example 14, when 
job 21 is made an immediate successor of jobs {1,...,6}. 

(2) An interesting property of 0* is the fact that it is not possible to take away 3 minimal elements 
from 0* and still remain 3-critical. Luckily, the elimination of e.g. jobs {13,14,15} preserves the 
property of being 3-critical. For induction proofs it would be very advantageous to establish the 
result, that for the problem instances considered here, either 3 minimal or 3 maximal elements can 
be taken away, such that the remaining poset is still 3-critical, except for the case that already a 
chain of blocks consisting of independent elements give the reduced schedule length. This is what 
will happen in the poset used in Example 13, regardless of how we take away 3 minimal or 
maximal elements. 

(3) Further 3-critical extensions 0*^ with i, j, k e M of 0* as considered are obtained by adding 
a chain of i blocks of 3 elements each, following jobs {9,10,13}, a similar chain of j blocks of 3 
elements each, following jobs {11,12,14} and further 3k independent elements, following jobs 7 
and 8 (independent from job 15). These posets have 15 + 3(i + j + k) elements and the associated 
optimal 3-machine unit time schedule length is 6 + (i + j + k). A bound contradiction for all these 
examples w.r.t. the schedule length 5 + i + j + k can be obtained completely analogously to the 
argumentation in Example 20. 



6. CONCLUDING REMARKS: 

The authors hope that this report on ongoing research will further stimulate work on a solution of 
the general m-machine unit time scheduling problem. The insights presented and the examples 
given might prove helpful in this respect. From the point of view of the authors, concentration on a 
characterization of m-critical posets, e.g. for m = 3, might be particularly worthwhile at the 
moment, also concerning a proof of membership to CO-NP, but even more for finding polynomial 
algorithms. There were given strong indications w.r.t. the special nature of such posets. Also, 
these structures really seem to help in proofs on the correctness of the proposed bounding 
systems, particularly again w.r.t. the 3-machine case. As was shown in the paper, a general answer 
for these steps would yield the general polynomial time solution procedure wanted. 

Presently, the two last authors are further looking into these questions, together with different 
co-workers, in particular also students from the University of Passau. In this context, special 
classes of posets are studied in more detail and the algorithmic procedures mentioned in this paper 
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are all encoded and tested in typical examples. Progress in all these aspects will be made available 
on request and will possibly also be published later at some appropriate point of time. 
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ON INVERSE-IMAGE OF NON -ORIENTED GRAPHS 



S. Perz and S. Rolewicz (Warszawa) 



By a non-oriented graph we shall call a pair G=<V/A>, where V is 
a finite set, and A is a family of two-elements subsets of V. 

The elements of V we shall call vertices, the elements of A we 
shall call edges. 

In sequel we shall consider only non-oriented graphs and we shall 
call them briefly graphs. 

Let U be a subset of V,U civ. We write A J ^ a family consisting of 
two-elements subsets of U belonging to A. 

Of course the pair 

G |u =<u ' A |u > 

is also a graph. We shall call the graph Gj^ a subgraph of the 
graph G. 

Let a graph G=<V,A> be given. A subgraph G 0 =<V 0 ' A 0 > is called a 
clique if G q is a complete graph, in other words A q is a set of 
all two-elements subsets of V Q . A subgraph G^=<V^,A^> is called 
an anticlique (or stable set) if G^ is an isolated graph, in other 
word A^ is empty. 

Let G=<V,A> be a graph. By a complement of the graph G we shall 
call the graph G=<V,A>, where A consist of those two-elements 
sets which does not belong to A. 

By a chromatic number x(G) of a graph G=<V,A> we shall call the 
minimal number of anticliques covering V. The maximal number of 
vertices in a clique contained in G we shall call the clique num- 
ber and we shall denote it o)(G) . 
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A graph G=<V,A> is called perfect if 



(1) 

for all Ucv. 



X (G a )=w(G a ) 



Now we consider two graphs G=<V,A> and G~ 1 =<W,D>. 



Let f be a function mapping W onto V. The graph G~^ we shall call 
an inverse image of the graph G if for arbitrary ,v 2 £V such 
that {v 1 ,v 2 }GA for arbitrary w^ ,w 2 such that ffw^)^^ f 2^ w 2^ =u 2' 
{w 1 ,w 2 }GD. 

_ 'I 

Of course for each vGV f (v) is a subset of W. Thus we can con- 



,-1 



-1 



f" 1 (v) =<f (v) ' D 



Since V is fi- 



sider a subgraph G 
nite we can numerate elements v n writing v^ r ... f v n . We shall brief*; 
ly write 



.-1 






D r“ 1 / v — A • • 

f (v ± ) 1 



Theorem 1 . A graph G ^=f ^ (G) is perfect if and only if the graphs 
G=<V,A>, G^=<V^,A^>, i=1,2,...,n are perfect. 

The special extremal cases of Theorem 1 are well known. Namely 
Theorem 1 are known when G^ i=1 , . . . ,n full graph (see [9], [10]) 
and when G^, i=1,2,...,n are isolated graphs (see for example 
[ 6 ]) . 

The authors do not know any paper containing the proof of Theorem 
1. However they estimate that the problem of priority of Theorem 1 
is not essential for them. Their idea is to present another ap- 
proach to the graph theory based on functional analysis, and a 
notion of norms. 

This approach was introduced by the first author [l 5 ] . It is a con- 
timation and developing of approach to the graph theory via con- 
vex polyhedra introduced by Fulkerson ([4], [5]) and Padberg 
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([11 1— [14]) . The first essential observation in this approach is 
that the problem of perfectness of the graph is much better fit- 
ted in the framework of the hypergraph theory than in the graph 
theory itself. 

We recall that a hypergraph is a pair G=<V,C>, when C is an ar- 
bitrary family of subsets of V. Let a graph G=<V,A> be given. 

By A c we denote the family of all subsets UcV such that G^ is 
a clique. Thus G c =<V,A c > is a hypergraph corresponding to the 
graph G. The family A c has the following properties 

(a) each one-element set belongs to A c 

(b) if a£A c and be a then b£A c 

(c) if all two-elements subsets of a set a belong to A , then 

It is easy to observe that each family satisfying (a) -(c) induces 
a graph G=<V,A> where A is defined as all two-elements set of A c . 
Therefore there is a one-to-one correspondance between the graph 
G=<V,A> and the hyper graph G c =<V,A c >. 

Let G=<V,A> be a graph let G=<V,A>be its complement. The both 
graphs induces the corresponding hypergraphs 

G =<V, A > and G =<V, A >. 
c c c c 

It is easy to verify that A c consist of all anticliques of A and 
such called transversality condition holds i.e. the cardinality 
of a common part of a and b is at most one: 

(1) #aflb^1 

for all aGA ,bGA . 

c c 

The set of all b satisfying (t) for all aEA we shall call the 

t r 

transversal set to A and denote by (A) . Of course 

Ac:((A) tr ) tr . 



( 2 ) 
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f r 1 f r 

If the equality holds i.e. A=((A) ) we shall call that the 

hypergraph G=<V,A> is conformal. It is easy to observe that a 
hypergraph is conformal if and only if (a) -(c) holds i.e. if it 
is generated by a certain graph. 

Let certain conformal hypergraph G c =<V,A c > be given. Let n be 
equal the cardinality of V, #V=n. We shall order to the hyper- 
graph G certain subset tl of R n . Namely we enumerate all ver- 
c ^ 

tices (elements of V) . Thus elements of IR can be considered 
as a real valued functions defined on V. The characteristic func- 
tion of a set a cV is a function 

1 for v. ea 

Xa (v i } = 

0 for v^a. 
fra 

By H we denote the set of 2 T functions consisting of those 
a 

function a that |a(v.) |=v (v.) . The set • The set 

l ^ ~a x a 

induces on the space consisting of vector of integer valued 

coefficients so called Konig norm in the honour of the fundament 
tal work of Konig (see [5], [6]) 

ll x ll A = inf {I i = 1 kil :x =Ii=iai a i' a i 6 & , o^-integers } 

and so called Minkowski norm 

I x| _=inf { V . I a . I :x=y . n ,. a . a . , a . G , a . -reals). 

A ^i=1 1 i‘ L l=^ l i' i l 

Of course the Minkowski norm is well determined on the whole R n 
and can be also defined there by the classical formula 

| x| A =inf { t>0:^f£conv GL } . 

Observe that the both norms satisfies the triangle inequality 
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l|x+y|l A * l|x|| A + ||y|| A 

|x*y| A £ j-x| A + |y| A . 

If x=0 then ||x|| A =0= |x | A and conversely if ||x|| A =0(or |x| A =0) then 
x=0. 

Moreover if |x|=( |x 1 | , . . . |x | ) then 

II l x l II A = ll x ll A ' I l x l l A =l x l A - 

Observe that 0^|x| A ^ ll x ll A - The Minkowski norm is homogeneous 

II tX H A = I I i x I A* 

Let a graph G=<V,A> be given. By G = <V,A > we denote the corres- 

c c 

ponding hypergraph, constructed in the way described previously. 

Let UcV be an arbitrary subset of V and let it^x^ denote the 
characteristic function of the set U. Observe that the clique 
number wtGjy) of the graph G| u is equal to 

a) (G I )=max #mOa 
,U a€A 

c 

=max (m,a) = max (ra,a) = |m| A 
a ea aGconv a 

where (m,a) denotes the inner product and |m| A denotes the norm 
conjugate to the Minkowski norm |x| A . 

Let G=<V,A> denote the complement of the graph G and let G c = 

<V,A c > denote the hypergraph corresponding to G Q . 

Let UczV and let n^Xy be a characteristic function of U. It is easy 
to observe that the chromatic number xtGjy) equal to the minimal 
covering of U by anticliques i.e. x(G| u )= ||in||“, where ||m||- denotes 
the Konig norm in G c =<V,A^> induced by A c . 

Thus a graph G=<V,A> is perfect if and only if for any character- 
istic function n^Xy we have 
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IMI A =N A * 

This equality leads to a series of equivalent conditions, proved 
by the first author ([15]). 

Theorem 2 . Let G=<V, A> be a given graph. Then the following con- 



ditions 


are equivalent 






(a) 


the graph G is perfect 






(b) 


the graph G is perfect 






(c) 


IMI A =M A 






(d) 


imi a =m a 






(e) 


MI a =I x I^ for all xez n 






(f) 


IMI A =l x l A for a11 xezI1 






(g) 


the Konig norm ||x||- is homogeneous 


for 


kez 


(h) 


the Konig norm ||:x|| A is homogeneous 


for 


kez 


(i) 


lim jj; ||kx||- is an integer 






(j) 


lim t- ||kx|| is an integer 






00 


the sets conv A, conv A are polar each 


to the 




second in 3R n . 







As a consequence of Theorem 2 we obtain 

Corollary 1 (see [1]). A graph G=<V,A> is perfect if and only if 
the polar set to the set conv'CL has only extremal point of the 
form where is equal either to 0 or to 1 or to -1. 

Proof. If the polar set to the set conv Os is conv OL then its 
extremal points belong to tX . Thus they are of requested form. 
Conversely if the point m of the requested form belongs to the 
polar set to conv CL then sup (m,x) £ 1 . Thus the support of m 
belongs to A and mE CL . X ^CL 

The characterisation of the perfect graph given by Corollary 1 
will be used in our proof of Theorem 1 . 

Before starting the proof we shall recall a classical construc- 
tion of product space with respect to a given space done by Day 
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[2 ] (see [3 ]) . 

Let (X, |[ H) be a Banach space of dimension n. The space X is 
called solid if for x=(x,j,...x n ) y= (y.| r . . .y ) the condition 
|y i | ^ | x i | i=1 , 2, . . .n implies that |J*y|| £ ||x||. By the construc- 
tion of the set CL it is easy to see that the space (3R n , | | A ) 

is a solid space. 

Let (X, || ||) be a solid Banach space. Let (X^, || || . ) i=1 , . . . ,n 

be a system of n Banach spaces. We construct a new space called 
the product of spaces X^ in the sense of X in the following way. 
We take the carthesian product X^ x . . . x X n and we equip it in 
the norm 

III (x r ...,x ) III = ||(||x II ||x II )|| . 

1 n 

The space (X^ x ... xx n ,||| ||| ) we shall denote briefly (X^) . 

X 

Let G= <V, A> , G=<V^,A^> denote the graphs described on the begin- 
ning of this paper. We construct a corresponding sets CL , CL 
i=1 . . .n. And we order to the each of those graph the correspond- 
ing Minkowski norm. In this way we obtain n+1 solid Banach spaces 
(X, III || j^) , (X^ /||| HJk ). The dimension of X is n. Now we shall con 

sider the product of 1 the spaces X^ with respect to the space X, 

(X.) and we shall show that the space ((X.) ,||| |||) is the space 

1 X 1 X 

generated by the graph G^=<W f D>. The proof will be based on the 

following classical theorem. 

Theorem 3 . Let (X f || || ) be an n-dimensional solid Banach space. 

Let ( X ^ , 1 1 ||^) i=1,...n be finite dimensional. A point x=(x,j / ,,x n 

is an extremal point of the unit ball j = {x: ||| x |[| ^ 1 } in the 

space ((X.) , HI |||) if and only if *X 

1 X 

(a) the point C || x^|| -j , . . . ,|| x | is an extremal point of the 

unit ball U x ={x: ||x|| ^1}of the space (X, || || ) 



and 
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(b) x i is an extremal point of the ball B^(||x^|| ) of the radius 

l|xj| in the space X^, i=1,2,...,n. 1 

i 

Proof. Let x=(x^...x n ) be an element of norm one, |||x||| =1 . If x 
is not an extremal point of U^ x ^ then there are two elements 

1 X 

u= (u<| , . . . u n ) and v=(v^, — v ) of norm one, |||u|||=1, |||v||| =1 such 

that x=^~^ and u4x=j=v. 



By triangle inequality 






i 



* i( 

- 2 k 



l u ±l 



and since Element (||xj| ) (||uj| ) and (||vj| ) has norm one in 

the space X, then we muit have 1 equality 1 



2 

2 



^11 /II v ill , ) • 



Now there are two possibilities, either 

(i) there is i such that ||xj| =|=ll u ill • In this case (a) does not 
hold 1 1 

or 

Ui) IM Hlu ± || Hlv ± || . 

Since x4=u, in the second case there is an index i such that 

o 

x. +u. and it implies that (b) does not hold. 

0 o 

Conversely suppose now that (a) does not hold i.e. (||xJ| ,... 

1 

|| x || ) is not an extremal point of the unit ball in U x . 

It means that there are two systems of nonnegative numbers 

(a 1 ...a ), (b 1 ,...b ) belonging to the unit ball U v such that 

1 n i n a 

(a 1 , . . .a ) + (|| xJ| , x || ) and 
i n i 1 n n 

a . +b . 

|| x . 1 1 n 1 1=1 , . . . ,n . 

11 i. 2 
l 

x . x . 

Let u =a.|| n , v i =b ri - x n if II X II 4=0. If II X II =0 we put u = 

11 i M . 11 i N . i i 
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v ± =0. It is easy to observe that u=(u.j , . . . , u ) , v=(v 1 ,..., v ) 

are different than x=(x ,...,x ) the norm |||u||| =|||v|||=i and 
u+v . . n 

moreover x — j- i.e. x is not an extremal point of the unit ball 

u (x,) • 



Suppose now that (b) does not hold i.e. there is an index i 

o 

such that x ± is not an extremal point of the ball B. (||x. || ). 

o 1 o 1 o i 

Thus there are two elements of x. . u. and v. such that ° 



1 o 1 o 



I X i II =11 u ± II =11 v ± II , 

° \> ° *0 ° i o 



X . =- 

x o 



u +v. 
io *0 



and x. =|=u^ . 
o o 



x. for i4=i 
1 1 o 

Let u.= . . 

l u. for 1=1 
1 o 

o 



for i+i 
1 1 o 



v . = 

1 v. 
i 



Thus X = ~jr~ r X^U 



I II u III = I = III v l II and x i s not -an extremal point of U 
As a consequence of Theorem 3 we obtain 



(x) 



x . 

l 



-1 

Theorem 4. Let G=<V / A> be a given graph. Let G =<W,D> be an in- 
verse image of the graph G. Let V i =f (v i ) and A j_ =D | v . Let (X,|| || ) 
be a solid Banach space corresponding to the graph G=^V,A>. Let 
(X^ ,|| ||^) be Banach spaces corresponding to the graphs G i =<V i ,A i >. 
Let (Y,|| || Y ) be a Banach space corresponding to the inverse image 
G -1 =<W,D>. Then (Y,|| H y ) = ( (X ± ) ,||| |||). 

X 

Proof. Let d be an arbitrary element of D^. Let m= (m^ , . . . ,m n ) be 

an arbitrary element of i.e. such an element that |m | = ( | m-j | , . . . 

| m n |) is a characteristic function of d. By the definitions me . 

Since dnx.^eA.j^ we have that x^mj^ belongs to tZ- i and 1 1 x^| | =1. 

Since by the definition of the inverse image (|| x^ || , ...||x || f'e VL 

we obtain that |||m||| <; 1 . On the other hand let x=(x v ..x n f be 

an extremal point of the unit ball U /v . . 

(X . ) 




222 



Now we shall use Theorem 3. By property (a) (||xJ| , ...||x || ) 

1 n. 

is an extremal point of the unit ball of the space X. Thus 
1 1 x . II is equal either 0 other 1 and the support of (||xJ| 
llxj 1 ) belongs to A. By (b) x i is an extremal point of the 
unit n ball of the space X.. Therefore x.=(x. .) where x. . is 

l li, j ± , j 

either equal 0 or 1 or -1 and the support of x^ belongs to A^. 
Those conditions and the definition of D and J) implies that 
x€ >2) . Since the unit ball in the finite dimensional space is 
uniquely determined by its extremal point Theorem 4 holds. 

For the proof of Theorem 1 we need a following classical 

Theorem 5 . Let (X,|| || ) , (X ± , || || ± ) i=1,2...n be finite-dimen- 

sional Banach spaces. Let dim X=n and let X be a solid space. 

Let (X, HI |||) be the product of the spaces (X, || ||^) i=1,2...n 

in the sense of space (X,|| || ) . Then the conjugate space has 

norm given by formula 

(5) nifiir- ii dif .ir ,...iif n n*)ii* 

1 1 n n 

where f = ( f 1 . . . f n ) , f i £ ( x *'ll ||*) and the norms ||||*|| || * i=1,2...n 

denotes the norms conjugate to the norms || || , || || ^ i=1,2,...,n. 

* 

Proof. Observe that for any f€Y and xGY 

I f (x) I^I^-] I f (x ± ) 1=^=., I I 
si.Mif.ir iix.ii ^ 

L i=i u i“ i 11 r 1 i 

^ ii oifjr — iif n ir)ir 

1 n 

II (||xJ| ,||x II )|| ^ 111 fill* lll x lll* 

1 1 n n * 

what implies that the norm of the functional f is not greater than 
||| f|||*. On the other hand let f=(f < |,...f ) be an arbitrary element 
of the conjugate space. By the definition of |||f||| there is a 
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system of nonnegative numbers (a^ f a 2 .*.a n ) such that || (a^ , . . . , 
a n )l|=1 and ||| f ||| M^ll ^11 .*a.. 

Now for each i we shall take x. such that llx.ll =a. and f.(x.) = 

II f ill. a ± . 

Thus 1 

III f llT = h=1 f i (x i )=f (x) 

and this shows that |||f|||* is attained. 

Proof of Theorem 1 . Let X denote the solid Banach space genera- 
ted by the graph G=<V,A>. Let denote the Banach space genera- 
ted by the graphs G.=<V.,A.>. Since those graphs are per feet , by 
Corollary 1 the unit balls of the conjugate spaces X , X^ i=1,2...n 
has only points of the form {e^} where is equal either zero or 
1 or -1 . 

Thus by Theorem 4 the extremal points of the ball in the product 
of X. with respect to X are of this same form. Thus by Corollary 

1 -i 

1 the graph G =<W,D> is perfect. 
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Existence of Optimal Trajectory of Mayer 
Problem for Neutral Functional Differential Inclusions 

Michai Kisielewicz 
Zielona Gora, Poland 



Introduction . Existence theorems for optimal control problems 
for systems described by neutral functional-differential inclusions 
can be obtained as a consequence of the compactness of the set of 
attainable trajectories and lower or upper semi-continuities given 
functionals. Therefore, we begin our lecture with some sufficient 
conditions for the compactness of the set T(D,F,A,B) of all abso- 
lutely continuous functions x:[a-r,cr+a] - 3R n satisfying the follow- 
ing conditions 

f x ( t) GF (t ,x t , x t ) for a. e. te[a,cr+a] 

(1) < (t,x t )£A for te[a,a+a] 

l (a,x a ,a+a,x a+a )£B, 

where D,A,B are nonempty subsets of Rxc ( [ -r,0] ,]R n ) xL ( t -r ,0] ,]R n ) , 
RxAC([-r,0] ,}R n ) and Rxc([-r,0] ,]R n ) xRxc([-r,0] ,]R n ) , respectively 
and F is a set-valued function defined on D with values in the 
space ConvOR n ) of all nonempty compact and convex subsets of ]R n . 
Here x fc and x t denote for given x:[a-r,cr+a] +IR n and a fixed 
t£[a,a+al mappings defined on [-r,0] by setting x t (s)=x(t+s) and 
x t (s)=x(t+s) for s£[ -r ,0] . 

In what follows we shall assume that sets D , A and B are such 
that pairs (D,A) and (A,B) have the following properties 
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(1) (D , A) is conformable, i.e., for every (t,x) , (t,y)GA 
we have (t,x,y)GD; 

(ii) (A,B)has a nonempty intersection property, i.e., it is 

such that a set K (A,B) ={ {x? [ a-r , a+a] ) E 3C r (A) : (t ,x fc ) 6A 
for t£[ a,a+a] , (a,x , a+a,x )GB} is nonempty. 

O ut9. 

Here and letter, 3£ r (A) = ag d <|5 o<a<3-a AC ( [ ^ -r / cr+a ] ,3R n ) , where A 

is assumed to have a connected projection on the real line, say 
equal to an interval [a, 3 ] such that $-a>r^O. As usual (see [1]) 
X r (A) is considered as a metric space with a metric d defined by 

(2) d[ (x-j ;[a 1 -r ,a^+a^ ] ) , (x 2 ; [ a 2 -r ,a 2 +a 2 ] ) ] = 

max C | cj - j — cr 2 I , | -a 2 | , J | x ^ — x^| | ] , 

oc 3 

where for x^AC ( [ a . -r , ] ,]R n ) , i=1,2, we define x A (t) =x.j_ (c^-r) 

for t6[ a-r ,o^-r), , (t) =x^ ( t) for t€[ a . -r , a . +a . ] , x A ( t) =x . (a . +a . ) 

for t€(o.+a.,p] and II xj| ^=1 xj(a-r) | +/J | xj(tf| dt . 

In a similar way we define a metric space (C (A),p) with 

C r (A)= a^a<3 o<a<3-o C ( [ a_r ' a+a] and p defined 

(3) p[ (x 1 ; [ o^r ,a 1 +a 1 ] ) , (x 2 ; [ a 2 ~ r f ° 2 +a 2 

max[ \a^-a 2 \ ,\ a^-a 2 \ , | x.,-x 2 | ] , 

where |*| denotes the supremum norm of C ( [ a-r , 3 ] ,]R n ) . 

Given a sequence { (x r ; [ ^ n " r / a^+a^]) ) °f 3€ r (A) is called d-weakly 
converging to (x;[a-r , a+a] ) if a n _>cr / a n ^ a an( ^ x n~*~ x ^ as n_>00 ' 
where -^denotes the weak convergence in AC ( [ a-r , 3 ] ,3R n ) . We will 
say that a set Sc 3£^(A) cC r (A) is p-closed or d-closed if it is 
a closed subset of a metric space (C r (A),p) or (3£ r (A),d), respec- 
tively. In a similar way we define p-compact subsets of C r (A). 
Finally, using the above mode of d-weak convergence in r (A) we 
can also define relatively d-weak sequentionally compact subsets 
of J£ r (A) • 
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1 . p-Compactness of Attainable Sets 



We shall present here some sufficient conditions for the non- 
emptyness and p-compactness of the attainable set T(D,F,A,B) con- 
taining all absolutely continuous functions (x; [ a-r , a+a ] ) e j£ r (A) 
satisfying conditions (1). We begin with the following definitions. 



We will say that F: D -► Conv (3R n ) is (u.s.c.w) on D if for every 

(t,u,v)ED and every sequence {(t n ,u n ,v )} of D such that 

1 1 -t I -► 0 , I u -u I 0 and v v as n->~, where |*| is the supremum 
* n 1 1 n 1 o n 1 1 o 

norm of C ( [ -r , 0 ] ,TR n ) and denotes the weak convergence in 

L ( [ -r ,0 ] ,]R n ) one has lim h(F(t ,u ,v ) , F (t , u , v) ) =0 with h(A,B) = 

n-*<» 

sup inf|a-b| for A , B ECon v (3R n ) . 
aEA bEB 

Given a set H c]R n the contingent cone T__(x) to H at xeH we 

n 

define by 



(4) 



V x) =e>o a>o o<h<a [ K (H " x)+E ^' 



where £ is a closed unit ball of lR n . 



Finally, given OEAC ( [ -r ,0 ] ,]R n ) , 1,M>0 and a nonempty set He iP 
such that 0(0) EH we define a set K^(0,H) by 

(5) ( $ , H ) = { V GAC ( [ -r , 0 ] ,]R n ) : ¥ (0) €H, / | ¥ (s) | ds<lw* M (p (E) ) 

E 

for every mesurable set Ec[-r,0]}, 

where ( 6) -6 ( 1+M) + sup { / | 0 (s) | ds : y (F) ^6 } . 

1M Fc:[-r,0] F 

By the above definitions it can be proved the following theorem 
( [ 2 ] , Theorem 3) . 



Theoreml . Let (D,A) be a conformable pair of nonempty 
subsets of RxC ( [ -r ,0] ,3R n ) xl ( [ -r ,0] ,3R n ) and RxAC ( [ -r ,0 ] ,3R n ) , respec- 
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tively and let B be a nonempty closed subset of 

RxC ( [-r,0] ,]R n ) xrxc ( [-r ,0] ,3R n ) . Suppose F: D - Conv 0R n ) is (u.s.c.w) 
and bounded on D. Assume A is closed in RxC ( [ -r ,0 ] ,lR n ) and relat- 
ively weak sequentionally compact in RxAC ( [-r , 0] f 3R n ) and has the 
connected projection, say equal to an interval [a, 3] on the real 
line such that 3-a>r>0. Let F , A and B be furthermore such that 
there are A q c A, (cr , $ ) EA q , a>r ,M>0 , 1>0 and a compact set H <=3R n such 
that $(0)GH, [a,CT+a]xK^($,H) cA q ,A o xA o cB,F (D) M^B and 
P(t, ’!',¥) nT t ,(¥ (0) )?0 for t€[ a , a+a ] and yGK 1 ($,H) then T (D,F,A,B) 
is a nonempty relatively d-weakly compact subset of 36^ (A), p- 
compact in C r (A). 



2. Existence of Optimal Trajectory 

We shall consider now the problem minimization of the functional 

I[x]=g(a,x ,a+a,x ) on the attainable set T (D,F, A,B) . Such prob- 
c a+a 

lem is called the Mayer problem for systems described by a neutral 
functional-differential inclusion x ( t) 6F ( t ,x^ ,x^) . It is denoted 
by JK D,F,A,B,g) . 

We shall prove now the following existence theorem for the 
Mayer problem defined above. 

Theorem2 . Let (D,A) be a conformable pair of nonempty 
subsets of Rxc([-r,0] ,K n ) xL([-r,0] ,]R n ) and RxAC ( [ -r ,0] ,]R n ) , res- 
pectively and let B a nonempty closed subset of 
RxC ([ -r,0] ,3R n ) xRxc ([ -r,0] ,]R n ) . Assume F,A and B are such as in 
Theorem 1. Then for every lower semicontinuous function g:B-*R 
there is an optimal trajectory of the Mayer problem ^(D ,F , A,B ,g) . 

Proof . Let us observe that A is compact in RxC ( [ -r ,0] ,3R n ) . 

By Theorem 1, T (D,F, A,B) is a nonempty relatively d-weakly sequen- 
tionally compact subset of 3(^(A) , p-compact in C r (A) . Therefore t 
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a pair (A,B) has a nonempty intersection property. Then (A X A) flB 
is a nonempty compact subset of R X C ( [ -r ,0] ,3R n ) X R X C ( [ -r ,0] ,]R n ) . 

Since g is lower semicontinuous on B then by Weierstrass theorem 
there is a y>0 such that inf {g (z) : zE (AxA) HB}^-y . Hence it follows 
that for every xET(D,F,A,B) we have l[x]^-y. 

Let i=inf { I [ x ] : xET (D,F , A,B) } . We have i^-y>-°°. Since T(D,F f A,B)^0 
then we also have i<+°°. Thus iE (-«>,+<») and therefore there exists 
a minimizing sequence, say { (x n ; [ a -r , a +a ] ) } for i, i.e., there 
is a sequence (x ) of T(D,F,A,B) such that i=jLim l[x ]. Since 
T (D,F,A,B) is relatively d-weakly sequentionally compact in 36 r (A) 
then there is a subsequence, say { (x ; [ a^-r / a ] <: +a ] c ] ) ) of 
{ (x n ;[a n ~r , a n +a n ] ) } d-weakly convergent to any (x; [ a-r , a+a ] ) ET (D ,F , A,B) 
By the p-compactness of T(D,F,A,B) we can well assume that 

(a k' x a k ' a k +a k' x a k +a k ) ^ (CT,x a' a+a ' x a+a ) in RxC ^" r ' 0 ^ *» n ) x R x C([-r,0] ,]R n ) 

as k^°°. By the lower semicontinuity of g it implies 

I[x]=g(a,x ,a+a,x , )£lim inf g(a ,x* ,a,+a. ,x* )=i. Since 

0 CT+a k o k K k a k +a k 

(x; [ a-r , a+a] ) ET (D,F, A,B) , we have also l[x]^i. Therefore, l[x]=i 

and the proof is complete. 
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SMOOTH SOLUTIONS OF LINEAR EQUATIONS WITH 
SCALAR COEFFICIENTS IN A RIGHT INVERTIBLE OPERATOR 



D. Przeworska-Rolewicz (Warszawa) 



The purpose of the present paper is to examine the smoothness 
of solutions to linear equations with scalar coefficients in a 
right invertible operator. This leads to studies of some spaces 

oo 

bearing an analogy to C -spaces and their subspaces, in parti- 
cular to spaces of analytic functions. 



CONTENTS 

1 . Algebraic properties of D^-spaces 

2. Spaces of D-analytic elements 

3. Spaces of D-paraanalytic elements 

4. Systems with scalar coefficients 
References . 



1 . ALGEBRAIC PROPERTIES OF Ppp-SPACES 

Let X be a linear space over a field of scalars (for instance, 
3* =C or ?"=IR ) . By L(X) we denote the set of all linear operators 
with domains and ranges in X. In particular, L q (X) ={AEL (X) : dom A= 

X}. By R(X) we denote the set of all right invertible operators 
from L (X) . For a given DER(X) we shall denote by Jt D the set of 

all its right inverses and by 3^ - the set of all initial opera- 
tors , i . e . 

#T\ 

Theory of right invertible operators and its applications can 
be found in the book [4] of the present author. 
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= {Ren, Q (X) :DR=I } 

^ D ={F6L o (X) :F 2 =F,FX=ker D and 



rI# d 



FR=0) 



We shall assume here and in sequel that ker Dfi{0}, i.e. the opera- 
tor D under consideration is right invertible but not invertible. 
There is a one-to-one correspondance between sets and . 

Namely, an operator F is an initial operator for D if and only if 
there exists an RE^^ such that 



(1.1) F=I-RD on dom D. 



On the other hand, if FeL q (X) is any projection onto ker D then 

F is an initial operator for and the corresponding right inverse 

R=R -FR does not depend on the choice of a right inverse R . 
o o c ^ o 

An operator a EL q (X) is said to be a Volterra operator if I-\A 
is invertible for all \E(E. The set of all Volterra operators 
acting in L q (X) will denoted by V(X). 

Let ]N be the set of all positive integers. Write: 



(1.2) m =]N U {o } 

o 

(1.3) D 0 =x . D k =dom D k for k£]N , DeR(X) 

"• 4 > D ." k Do D k - 

Clearly, 



( 1 . 5 ) X=D z> D . id ... dD, z> . . . z> D . 

o 1 Jc °° 

If dom D^X then for k=0, 1 ,2 , . . 

The following spaces were distinguished by von Trotha [7] as 
canonical for a DeR(X) : 
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D „ itself; 

(1.6) s= i^ 0 her D 1 ; 

(1 * 7) Q R = i fl 0 Rlx ' rg ^ d - 

By definition, S is the set of all D-polynomials ; 

(1.8) S=lin{R k z:zGker D, keiNJ 

and this set is independent of the choice of ReJ£ D (cf. [4]). 

EXAMPLE 1.1 . Let X=C[a,b], D=^, R= , where t o e[a,b]. Then 
D,=C k [a,b] for keiN and D =c”[a,b]. ° 

Js. O oo L ' J 



DEFINITION 1 . 1 (von Trotha [7]). Let DeR(X) and let Re be 
arbitrarily fixed. The triple (X,D,R) is called a D-R space . 

Let Y be a subspace of X. The triple (Y,D| y ,R| y ) is a D-R sub- 
space if there is a subspace Y^c X such that Y^c dom D,RY C Y 
and DY .j c Y . If, in addition, Y^=Y, then Y is said to be D-R in- 



variant subspace. 



We shall recall without the proof the following 



THEOREM 1 ■ 1 (von Trotha [7]). Let DGR(x) and let ReJ? D . Then 

(i) S is the smallest D-R invariant subspace of X containing 
the kernel of D; 

(ii) Q r is the largest D-R invariant subspace of X such that D 
and R are mutually inverse and 

(1,9) Q R = iQo RlD » ={x6D » :RnDnx=x for a11 neiN}. 

Moreover, if Q o #{0> then dim Q=+~. 

K R 

(iii) ScD^, Q r cD m , SnQ R ={0} and S (J) Q R is a D-R invariant sub- 
space; 

(iv) D oo largest D-R invariant subspace of X. 
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THEOREM 1.2 . Suppose that DER(X) and F is an initial operator 
for D corresponding to an REj2 d (i.e. FR=0) . Then 

(1.10) Q R =Qp , where Qp= (xED^: FD n x=0 for xe3N Q }. 

Proof. Suppose that xEQ R . Then the Taylor Formula (cf. [4]) 
and Formula (1.8) together imply that for every nEU we have 

l £”q R k FD k x= ( I-R n D n ) x=x-R n D n x=0 . 

In particular, Fx=0. By an easy induction we find that R n FD n x=0 
for all nE]N , which implies that FD n x=D n R n FD n x=0 for all nE3N . 
Hence xEQ R . Conversely, suppose that xEQ R . Then FD n x=0 for all 
nC2N , which implies that for all nE]N x-R n D n x= (I-R n D n ) x= 
J^”qR^FD^x= 0 . Hence xEQ R . 

COROLLARY 1.1 . Suppose that D, R,F are as in Theorem 1.1. Then 
xEQ R if and only if xED to and D n xEker F for all nEIN . 

COROLLARY 1.2 . Suppose that X,D,R,F are as in Theorem 1.1. 
Write: 

°- R e% D Q R' HeV d V 

Then Q=Q <= . 

The space Q is called space of singular elements . 

PROPOSITION 1.1 . Suppose that X,D,R,F are as in Theorem 1.1 
and that 



^F,m ={xeD °° : n!m FDnx=0} for an • 

If x£Q_ then 
F,m 



rm-1 _>k„k ~ 

U=X “Z. k=0 R FD xE Q R = Qp 
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i.e. u is singular. 



Proof. By our assumption, since DF=0 and FR=0, we have for 
0 — j -m-1 

FD j u=FD j x-FD j I^”QR k FD k x=FD j x- ^ =0 FD j ~ k FD k X- ^“1 +1 FR k “ j FD k j 
=FD-’x-F 2 D^x=FD- I x-FD : ’x=0 

and for j^m 

FD ^ u=FD ^ x-FD ^ ^“^R k FD k x=FD ^ x- “^FD ^ " k FD k x=FD ^ x=0 . 



EXAMPLE 



1 .2 . Let X=C(IR), D=~:, R=/£, (Fx) (t)=x(0) for xeX. Then 



F 

e 



is gn initial operator for D corresponding to R. Define: 

-1/t z j — Lr\ mu r~r* °° f 1 v-> -*»U n 4- v a v 



X(t) = 



and x (n) (0)=0 for all nE]N o . Hence x is a singular function, more 
precisely, xEQ R . 

Recall without proof the following 



for t^O and x(0)=0. Then xEC [-a, a] for an arbitrary aE3R + 



THEOREM 1 . 3 (cf. the author [4]). Let X be a linear space over 
an algebraically closed field F of scalars. Denote by SF [t] the 
set of all polynomials with scalar coefficients (i.e. with coef- 
ficients from £”) . Let P(t)E^"[t] and let 

r . 

P ( t) =n (t-t . ) ^ , where t.^t, for j^k, r- + ...+r =N. 

3-13 3 K in 

Suppose that DER(X) and R is a Volterra right inverse of D. Then 
the general solution of the equation 



(1.12) P (D) x=y , yEX 



is of the form 



(1.13) x=I j ^ 1 (I-t.R) (d jm RVz jm , where z jm €kerD) 



and the scalars d^ m are determined by decomposition of the rational 
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function 1/P(t) onto vulgar fractions: 






PROPOSITION 1.2 . Suppose that D£R(X) and R6^? D - If uGD k for 

a k€3N then 
° 

(i) R u6D, for mE]N (cf. Formula 1.3)? 

K+m 

(ii) (I-XR) -*uED k for XE 3 r ', j E2N , provided that REV(X)? 

( iii) (I-^R) _:l R n 'u=R m (I-\R) -:l uGD k+in for xeF, j ,mG]N o , provided 
that RGV(X) . 



Proof, (i) Since D (Ru) =DRu=uED k , we have Ru€D^. + ^ . By an easy 

induction we find that R m uED, , for mEU . (ii) Write: v=(I-\R) ^u. 

k+m 

Then u=v-XRv. Hence Du=Dv-\v and Dv=Du+\v. This implies that vED. 



Again, by an easy induction we find that D m v=£j™Q\ m ^D-*u for 
m^kElSf . Hence vED k . By an induction with respect to j we obtain 
(ii) . Points (i) and (ii) together imply (iii) . 



D J u for 



An imediate consequence of Theorem 1.3 and Proposition 1.1 is 



PROPOSITION 1.3 . Suppose that all assumptions of Theorem 1.3 
are satisfied. If yED k for a kE3N Q then every solution of Equation 
(1.12) belongs to D k+N _ n , hence to D^. 

Proof. Point (iii) of Proposition 1.1 implies that xED k+a , 
where a=£ ^ (r ..-1 ) ^ r j-n=N-n^O. By definition, D k+a <= D k for 

every k, aE3N o . 



COROLLARY 1.3 . Suppose that all assumptions of Theorem 1.3 are 
satisfied and that P(t) has single roots only (i.e. r^=. . .=r n =1 ) . 
If yED k for a kE3N o then every solution of Equation (1.12) belongs 
to D k (and, in general, do not belong to D k+1 ) . 

NOTE : Spaces D k for kE3N are not D-R invariant. 
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DEFINITION 1.2 (cf. von Trotha [7]). Let & be an algebraically 
closed field of scalars. Let U be a D-invariant subspace of X, 
i.e. DU cu. Then U is said to be D-closed if for every ’P{t)e^ r [t] 
we have P(D)xEU if and only if xEU. The set 

(1.15) H D (U) = {xGX: p(t) |^r [t] P(D)xeU> 

is called a D-hull of U. 

Clearly, a D-R invariant subspace UcX is D-closed if and only 
if H d (U)=U. 

Observe that H D (0) is the space of all solutions, of all homo- 
geneous equations P(D)x=0 with scalar coefficients. 

We recall, again without proof, the following 

THEOREM 1 . 4 (cf. von Trotha [7]). Suppose that DER(X) and 
REV(X) (1 . Then for every D-R invariant subspaces U,V cX we 

have : 

(i) U cH d (U)=H d [H d (U) ] . Moreover, H D (U) is a D-R invariant sub- 
space of D^; 

(ii) H d (U+V)=H d (U)+H d (V) and H D (UnV)=H D (U) HH d (V) ; 

(iii) Rational functions of R preserve H D (U); 

(iv) H D (U) is t ^ ie smallest extension space of U where every 
equation with scalar coefficients P (D) x=yEH D (U) has a 
solution and all solutions belong to H D (U); 

(v) H d (S)=H d (0); 

(vi) H d (Q r )=Q r © h d (0) . 

NOTE : Spaces D^ for kEIN are not D-closed. 

THEOREM 1.5 . Suppose that all assumptions of Theorem 1.3 are 
satisfied. Then H^(D ) =D , i.e. D is D-closed. In other words: 

JJ oo oo oo 
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if z/EZ^ and x satisfies Equation (1.12) then #E0 m . 

Proof. By Point (iv) of Theorem 1.1, is the largest D-R 
invariant subspace of X. By Point (iii) of Theorem 1.4 rational 
functions of R preserve . Assume that y£ . Observe that any 
constant, i.e. an element zE kerD belongs to . This, Proposition 
1.2 (iii) and Proposition 1.3 together imply that xED^ . Point 
(iv) of Theorem 1.4 implies that H^D ) =D . Hence D is D-closed. 

JJ CO OO CO 

We should point out that D is the largest D-closed subspace 
of X. If xeD then x is said to be smooth. 



Recall that x is an exponential for D if 0^x n Eker (D-a X) for 

A A 

a (cf . the author [ 4] ) . 



PROPOSITION 1.4 . Let X be a linear space over a field 3^ of 
scalars and let DeR(X) . Suppose that F is an initial operator 
for D corresponding to R and • Then 



(i) all eigenvectors of R belong to Q R , provided that they 
exist ; 

(ii) x eQr> is an exponential if and only if x Eker(i-XR) c= 

A. K A K 

(i.e. if x is an eigenvector for R corresponding to the 
A 

eigenvalue 1 /\ , which belongs to Q R ) ; 

(iii) if an exponential x then R n FD n x =^ n R n z for all nE3N , 

A r K A O 

where 0^z=Fx EkerD; 

A. 

(iv) if ker (I~aR) = {0} then Fx ^0 for any exponential x^ ; 

A A 

(v) if ker (1 -a.R) ^{0} then Fx =0 for any exponential x . 

A A 



Proof. Points (i)-(iii) have been proved by von Trotha [7]. 

(iv) Suppose that x is an exponential, i.e. O^x Eker (D-a I). Then 
A. X 

D (I-\R) x = (D-a I ) x =0 which implies (I-aR)x =z, where zekerD and 
XX X 

z=Fz=F ( I~ aR) x =Fx -FRx =Fx , for FR=0. Suppose that Fx =0. Then 
X X X X X 

we have (I-\R)x =z=Fx which implies that x Eker (I-aR) ={0} . Thus 
A A A 

x =0, which contradicts our assumption. Hence Fx ^0. Point (v) 

A A 

follows from (i) and (ii) , because in this case O^x Eker(I-AR) c: 

n ^ 

Q n implies FD x =0 for all nE^N , in particular Fx =0. 

K A o \ 
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This leads us to the following 

DEFINITION 1.3 . Suppose that DER(X) , F is an initial operator 
for D corresponding to an RE Q and x is an exponential. If 

D A, 

1/\ is a regular value of R then Fx^O and x^ is a regular expo- 
nential . If 1/\ is an eigenvalue of R then x GQ D , hence Fx =0 

A. R A, 

and x^ is a singular exponential . 

EXAMPLE 1.3 . Let X=C[0,1], D=^, (Fx) ( t) =j[x (0) +X ( 1 ) ] for xGX. 

Then F is an initial operator for D corresponding to a right in- 

verse R= f =i( f +flf) . The right inverse R is not a Volterra 

•'o2 ; o2 ; 0‘'1 ^ ^ 

operator and has X as an eigenvalue if Fe A =0 (cf. [4]). We find: 

0=Fe^ t= “-( 1 +e^) f i.e. e^=-1 . If y=JR , there is no XE3R satisfying 

^ x t 

this equality. Hence REV(X) and functions e are regular expo- 
nential for all XE1R . If then the equation e^=-1 has infi- 

nitely many roots =-7Ti +2krri , where kEZ . All numbers 1 /X v are 

* ^vt K 

eigenvalues of R. Therefore all functions e (kEZ) are singular 

exponentials. If X^X-^ then e^ is a regular exponential. 

EXAMPLE 1.4 . Let X and D be defined as in Example 1.3. Let 
(Fx) (t)=x(0)+x* (t) . In a similar manner as in the preceeding 
example we conclude that the corresponding right inverse is 
(Rx) ( t) =/ qX (s) ds-x ( 1 ) and that R has a real eigenvalue^?^, namely , 
a root of the equation 1 +\e^=0. Hence the function e ° is a sin- 
gular exponential in the space X considered over the field of 
reals . 



2. SPACES OF D-ANALYTIC ELEMENTS 



Here and sequel X will a complete linear metric space with 
F-norm || || and y =(C , if it is not stated otherwise. 

THEOREM 2.1 . Let DER(X) . Write: 

A r ( D ) = 

={xED :V ~ Rn FD n x is convergent and x=Y ~ R n FD n x} 

<*> ‘"11=0 t *n=o 



( 2 . 1 ) 




where F is an initial operator for D corresponding to an Re 
Then 



(2.2) A„(D)={xeD :lim R n D n x=0 } cD . 

K oo oo 

n 

Proof. Write: 

(2.3) Q n = ^kIo RkFC)k ' p n =RllDn for ne3N • 

By the Taylor Formula for an arbitrary xED^ and for nE3N we have 

Q„x=^“ 1 l R k FD k x=x-R n D n x=x-P x=(I-P )x. 
n k=0 n n 

Hence 

(2.4) x=P x+Q x for all nEIN and xED . 

n n oo 

Moreover , both Q n and P n are projections. Indeed, P^= (R n D n ) (R n D n ) 

R n (D n R n )D n =R n D n =P r , and Q^=(I-P ) 2 =I-P =Q on D (n£]N) . 

n n n n n °° 

Suppose now, that lim R n D n x=0. Then lim Q x=lim(I-P )x=lim 

n-^oo n-* 00 n->~ n->-oo 

(x-R n D n x) =x-lim R n D n x=x, which implies that the series Y ~ R n FD n x 

r z n=0 

n * C ° oo 

is convergent and that x=£ n ~ 0 R n FD n x. We therefore conclude that 
xEA R (D) . 

Conversely, if xEA_,(D) then the series Y “ R n FD n x is conver- 
k n— u 

gent to x. Hence, by the Taylor Formula 

lim R n D n x=lim (x-Tf - ( lR k FD k x) =x-lim y, n ~lR k FD k x=x-x=0 . 
^k=0 _ L k=0 

n->«> n**’ 0 ® n, ->-oo 



Write: 

(2.5) 



A(D) =Re U J? D V D) - 



Clearly, by (1.8) we have 
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The set A(D) is said to be the space of D-analytic elements 
(cf . the author [5]) . 

THEOREM 2.2. Let DER(X) . If RE is continuous then A„(D) is 

— — — — — L) K 

a D-R invariant subspace (cf. [5]). 

Proof. Write: Y=A R (D) . We have to prove that Y cdom D, DYcY, 

RY cY. By definition, Y c c dom D. Let yEY be arbitrarily fixed. 
Then yep^, DyeD^, RyeD^ and ^ k ” 0 R k FD k Dy=y k ” 0 R k FD k+1 y=I j ” 1 R j " 1 FD j y= 
DRj j“ 1 R- , FD“ , y=D^j~ 1 R“ , FD :, y=D(y-Fy) = Dy for DF=0. Hence DyEY. Similarly, 

I k ” 0 R k FD k Ry=FR+ \ fc “ 1 R k FD k " 1 Y= I j j + W =r£ j ” Q R j FD j y=Ry 

for FR=0. Hence RyEY. The arbitrariness of ycY implies that 
DYcY, RYcY. 

THEOREM 2.3 . Suppose that X is a Banach space, DER(X) and 
there is a quasinilpotent REj^ d - Then 

(i) H [A_(D) ]=A_(D) cA(D), hence A 0 (D) is D-closed; 

UK K K 

(ii) H d (0) cA r (D) c A (D) . 

In other words: all solutions of linear* equations in D with 
scalar coefficients and with the right hand side belonging to 
Ar(D) (hence D-analytic ) belong to Ar(D) s hence are D-analytic . 

In particular , all solutions of linear homogeneous equations in 
D with scalar coefficients are D-analytic (cf. also [5]). 

Proof, (i) By Theorem 1.4 and Formula (2.6), S cA R (D) c 
H d [A r (D) ] and the D-hull of A R (D) is preserved by rational func- 
tions of R. Consider the equation P (D) x=yEA R (D) , where P(t) is 
a polynomial in D with scalar coefficients. Since yEA R (D) , we 
have y=£ n ~ 0 R n FD n y . By our assumptions, R is quasinilpotent, 
hence a Volterra operator. Thus, by Theorem 1.3, any solution 
of the equation P(D)x=y can be written in the form (1.13). 
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Elements of the form R-^z, zEker D, belong to S, hence to 

A n (D) . Thus, in order to prove our theorem, we have to show 
K -1 

that RuGA R (D) , (I-\R) u6A R (D) , whenever uEA R (D) , XE( E. Having 

already proved this fact, by an easy induction we obtain that 

any solution of the equation P(D)x=y belongs to A R (D) , what 

means that A R (D) =H^[A R (D) ]. Suppose then that uEA R (D) , i.e. u= 

Y ” : R n FD n u. By Theorem 2.2, A„(D) is a D-R invariant subspace, 
n-u k <j 

Thus RuEA R (D) . Let now v=(I-R) u. Then, since FR=0, we find 

i n : 0 RnFDn v= i n : 0 RnFDn ( i - ^ r> - 1 u= 

= L” Rn FD n y :_X m R m u=L” A m L“ R n FD n R m u= 

L n=0 £ m=0 L m=0 L n=0 

r> oo rn ti oo r» n m 

= L-n^ L-r> R FD R u = 

L m=0 L n=0 

v 00 r^ n rnT> n , v 00 -v m v oo _n„n_m 

=y n=0 R fD u +I m=1 X l n=0 R FD R u= 

=U + L : 1 X m [ r: ( ! R n FR m - n u+ L : m R n FD n - m u ] = 

^m=1 ^n=0 L n=m 

, r 00 ^ m_n__n-m n oo m r oo m+ j 

=u+ L,-l^ R FD U=U +L> )* _n R FD j U= 

Zj m=1 L m=1 j =0 

m W 5in ;, A» u .i m ; 0 x” R “u. 

= (I-\R) ~ 1 u— V / 

what completes the proof of (i) . 

(ii) By (2.6) we have ScA^(D). This implies that 

h d ( 0 )=h d ( s) cH d Ca r ( d) ]=A r (D) cA (°) • 

PROPOSITION 2.1 . Let DGR(X) and R Then 

(2.7) Q R nA R (D)={0}. 

Proof. Suppose that xEQ R fiA R (D) . Then for every n^1 we have 
R n D n x=x. On the other hand, since xEA^iD) , by Theorem 2.1 we 

K. 
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find 0=lim R n D n x=x. This implies Formula (2.7) . 
n-*°° 

PROPOSITION 2.2 . Suppose that DER(X) . F is an initial opera- 
tor for D corresponding to a continuous right inverse R. Then 
R is a Vol terra operator on A R (D) . 

Proof. Write:* Y=A R (D) . Since R is continuous, we have RYcY. 
Suppose that R is not Volterra on Y. Proposition 1.3 implies 
that there exists a X^O and an exponential x^Gker(I-\R) cQ R . 

This leads to a contradiction, since x^O and Q R flA R (D) ={0} . Hence 
REV(Y) . 

Let X be a Banach space. Denote by QN(X) the set of all quasi- 
nilpotent operators belonging to L q (X) . Evidently, QN(X) cV(X). 

If the space X is over (E then we have QN(X)=V(X). This implies the 
following 

PROPOSITION 2.3 . Let X=C[0,1] over (C. Then the only quasinil- 
potent right inverse to the operator D=d/dt are of the form R=/^, 
where aE [ 0 , 1 ] . 

Proof. The only continuous right inverses of D in the space 
C[0,1 ] over <E are of the form R =/^, where aE[0,1] (cf. the author 

u 

and S . Rolewicz [ 6 ] ) . 

In the Banach spaces over reals we have QN(X)^V(X). Indeed, if 
we consider the space C[0,1] over 3R and the operator R defined in 
Example 1.3, then REV(X) but R^QN(X). However, if we consider the 
space Y=X+iX and an extension of AEL(X) defined as follows 

Ay=Ax^+i Ax 2 for y=x 1 +ix 2 , where x^ ,x 2 EdomA 



then we conclude that QN(X)=V(Y). 
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EXAMPLE 



2.1 . Suppose that X=C[1, + «>), D=t^-, (.Rx) ( t) = 



(s)ds, (Fx) ( t) =x ( 1 ) for xeX. Then Re^ D nV(X) and 

xgA d (D) if and only if x(t)=Y ~ *! -■-— (lnt) n . 

k L, n = u n i 



EXAMPLE 2.2 . Let X=C [0 
corresponding right inverse 



,1], D=^, (Fx) (t)=/^x(s)ds. 
se is (Rx) (t) = / qX (s) ds-/^( 1 - 



Then the 
-s) x (s) ds . 



X t 1 X 

Since 0=Fe =— (e -1) and \^0, we conclude that in the space 

A. 

C[0,1 ] over reals there is no eigenvalue of R. Hence R is a Vol- 

terra operator and all functions e^' t eA R (D) . However, in the space 

C[0,1] over (C the operator R has eigenvalues 1 /\ v , where \,= 27 ri, 

X k t K K 



kez . Therefore Rj£V(X) and e 



£A r (D) . 



EXAMPLE 2.3 . Suppose that X=(s) over 3P (where (Cor 3^=1R) 

with the convergence on each coordinate. Write: x={x n ) where 

nelN , 6 =1 for m=n and 6 =0 otherwise (m,neZ) , $={5 }. Con- 

m, n m, n l n 

sider the operator of forward s hift : D (x^ }= ( x n+ -j } . Then the oper- 
ator D is right invertible, ker D consists of all sequences c« 6 r 

where ce A right inverse is R{x }={x .} (where we admit x^ ,=0 

n n— I n-K 

if n^k for all x={x }GX) i.e. the operator of backward shift . By 

k ^ 

definition, R {x }={x , } for kelN . The corresponding initial 

n n-K . k k 

operator F is defined as F{x n )=x^ 6 . Then FD { x n ^^k+l R FD 

^ x n> =x k+1 Rk5=x k+1 {6 n-k,1> for ke3N c and 

Q N< x n> = Si RkFD ^ x n>=£o x k + 1 <Vk,1 >= 

= {x^ , x 2 , • • • , x^. ,0,0, • • • } 






lim R D {x n }=lim R {x^ +n }= {0 , . . . ,0 , x N+1 , x N+2 , . . . } . 
n-oo N->°° ^ * 

N coordinates 



N N 

By our assumptions, lim R D {x }=0 if and only if the n-th 
N N M_.ro n 

coordinate of R D {x } tends to zero for all ne3N as N tends 

n 

to infinity. Hence 

lim R N D N {x n )=0 and lim Q N { x n }=I^Q R k FD k {x n >={x n } . 

N - >c ° N ■ >o ° 
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The arbitrariness of xeX implies 
A r (D)=X. 

Proposition 2.2 implies that R£V(X) . It is easy to verify that 
exponentials are of the form: c{x n ^ , where c,^e 



EXAMPLE 2.4 . Suppose that X, D , R are defined as in Example 2.3. 

Consider an initial operator {x n }= (x^ +X£> 6 . The corresponding 

right inverse is R 1 {x }=(R-F,R){x }={x 1 }-x 1 5- It is easy to 

i n n *1 * < ^ ^ 

verify that all elements of the form x = x* {\ } belong to 

A, I 

ker(I-A.R^). We have two cases: 

1° x^=0. This implies x=0; 



2° x^O. Then A=-1 is an eigenvalue of R^ . Hence R^£V(X) 



and A 0 (D)^X. 
R 1 

seen that Q_ 

K i 

Indeed, 



By Proposition 1.3, ker(I+R^) cQ R . It can be easily 
consists of all elements of the firm x=x,j { (-1 ) n } . 



Q R., ={x6X: kIo FD X=0 >=^ eX: kL (X k + 1 +x k + 2 ) 6=0 > 



which implies x =-x . = 

c n n-1 



.=(-1 ) n-1 x 1 . Thus 



Q =ker (I+R 1 ) . 

R 1 ? 

PROPOSITION 2.4 . Suppose that DGR(X) , ker D^{0}, F is an ini- 
tial operator corresponding to a continuous Volterra right inver- 
se R and {Sj^hElR is the family of D-shifts induced by R, i.e. 

S, e F=e^ h e F, where e =(I-A.R) \ for all AGC for all \e(E, helR 

n A. A, a. 

(cf. the author [3], also [4]). Then all right inverses R^=R-FR^ 
are Volterra operators on A R (D) , where F^=FS^ for helR + . 



Proof. By Proposition 2.2, R is Volterra on A^(D) . In order 

to prove that all R^ are Volterra operators, we have to show 

that F^e^ (z) ^0 fcurO^zeker d and helR + , O^AGCE (cf. the author [4] 

p. 502) . Indeed, let heiR be arbitrarily fixed, O^zGker D,0 • 

Then F,e_ (z)=FS u e. (z)=Fe X e. (z)=e^ h Fe^ (z)=e^ h z^0. 
h X h A. A. A 
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PROPOSITION 2.5 . Suppose that DER(X) , ker D^{0}, F is an ini- 
tial operator corresponding to a continuous right inverse and 
{Sh >hGiR is the famil y of R-shifts (cf. the author [3], also 
[4]). Then all right inverses R h =R-F^R are Volterra operators 
on A r (D) , where F^=FS^ (hElR + ) . 

Proof. Suppose that there exists an h Q ^0 such that R^ is 
not a Volterra operator on A (D) . This means that there exists 

K. 

a X^ t^O and a yEA R (D) such that 0^yEker(I-\^ R^ ) c:Q R . This im- 

pliei that Dy=A, h y and Fy=0 (by Proposition ° ° 2.1). Since 

R, =R-F, R=R-FS, °R, we find 
h_ 

o o o 

(I-\ R)y=X h FS h Ry=FS RDy=FS h (I-F)y=FS h y. 
n o o n o h o o o 

Hence 0=F^ =FS^ y=(I-X h R)y, which contradicts our assumption that 
R is Volte?ra °operator on A R (D) . Thus all R^ are Volterra oper- 
ators on A_ (D) . 

K. 

3. SPACES OF D-PARAANALYTIC ELEMENTS 



In the present section we shall look for possible extensions 
of spaces of D-analytic elements which are D-closed. 

THEOREM 3.1 . Suppose that X is Banach space (over (C) . DER(X) 
and there is a quasinilpotent RG^ Q . Then 

(3.1) H d [A r (D) <±) Q r ]=A r (D) Q q r 

i.e. the set A R (D) @ Q R is D-closed. 

In other words: all solutions of linear equations in D with 
scalar coefficients and with right hand side which is a sum of 
a D-analytic element and a singular element 3 are again sums of 
a D-analytic element and a singular element . 
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Proof. Theorem 2.3 (ii) implies that H (D) c A n (D) . This, Theo- 

U K 

rem 2.3 (i) , Theorem 1.4 (ii) and (vi) , together imply that 



H d [A r (D) +q r ]=h d [A r (D) ]+H d (Q r ) =A r (D) +Q r +h d (D) 

=A r ( D ) + Qr, 

for both, A_ (D) and Q , are D-R invariant (Theorem 2.2 and 2.4). 

K K 

Proposition 2.1 implies that the sum A R (D)+Q R is direct. 

Write : 

(3.2) A^(D) = {x 6D slim R n D n x=x} for RE 

K 00 D 

n->°° 

Formulae (2.4) may suggest that any element of can be repre- 
sented as a sum of two elements which belong to A(D) and A^(D)= 

A^(D)/ respectively (not necessarilv in a unique way). We 
RE C&n K 

shall see that the situation in D is different. 

oo 



THEOREM 3.2 . Suppose that DER(X) and F is an initial operator 
for D corresponding to an REj? d - Then 

(i) Q R c a r (D) c 



(ii) if R is continuous then A*(D)=Q 0 , 

n v\ tv 

have lim R n D n x=x then R D x=x for all nE3N . 



i.e. if for an x® we 



n-»co 



Proof, (i) By definition, if xEQ_ then R n D n x=x for all nElN . 
n n 0 ^ 

Hence lim R D x=x and xEA^(D). (ii) Let RE be continuous. Write 

0 n+oo K D 

Y=A r (D 1 . Thus Y is a D-R invariant subspace. Indeed, if yeY is ar 
bitrarily fixed then 



limR n D n Dy=lim R n D n+1 y=DRlim R n D n+1 y=Dlim R n+1 D n+1 y=Dy; 
n-*°° n-*«> n->°° n-*«> 

lim R n D n y=lim R n D n y=Rlim R n-1 D n-1 y=Ry • 
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Thus Dy, Ry€Y . The arbitrariness of yGY implies DY cY, RYcY. By 
definition, for arbitrary xEY we have 

(3.3) lim [ k 2 0 R k FD k x=0. 

n-*°° 

Indeed, by Taylor Formula, 

lim £ k ” 0 R k FD k x=lim(x-R n+1 D n+1 x)=x-lim R n+1 D n+1 x=0. 
n->°° n-*-«> n-oo 

Suppose now that x^O and xeY^Q-. This means that there exists 

R 

an index n such that FD n x^O. Denote by n Q the minimal index with 
this property. This, and Formula (3.4) together imply that 

0=lim Y. n n R k FD k x=R °FD °x+lim Y " , . R k FD k x 

4 k=0 L k=n +1 

n-*°° n-oo o 

n n 

=R °FD °x. 

n n n n 

Hence FD °x=D °(R °FD °x)=0, which n contradicts our assumption that 

n_ is a minimal index such that FD °x=0. We therefore conclude that 

° 0 
x=0 and A R (D)=Q R . 

Observe that already distinguished D-closed subspaces of D^: 

Ar( D ), Q r , A r (D) ©Q r 

have the common property: 

(3.5) lim R n FD n x=0 

n-*°° 

for x belonging to these spaces. Indeed, if xEA R (D) then the series 

R n FD n x is convergent. Hence its n-th term R n FD n x tends to 

zero as n-*«. if x£Q D then FD n x=0 for nE3N . Hence R n FD n x=0 for 

R O 

n63N Q . If xGA r (D) © Q r then x=u+v, where u£A R (D) , v£Q R and 
lim R n FD n x=lim R n FD n u+lim R n FD n v=0 . 
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Another example with this property is following 

EXAMPLE 3.1 . Let DeR(X) and RE^ D be given. Let Y=X^X 2 , where 

X 1 =A d (D) , X«=Q . For an A EL (X) and x 1f x 9 Edom A we admit: A(x^ f x 2 )= 
I R 2 R I Z \ c. 

(Ax 1f Ax 2 ). Let 0 ^x 1 €A r (D), O#x 2 03 r . Then (x^x^j^O and 

lim R n FD n (x 1 ,x 2 )=(lim R n FD n x 1f lim R n FD n x 2 )=0. 
n->oo n-*°° n-*« 

Observe that in this case we have 

lim R n D n (x 1 ,x 2 )=(lim R n D n x 1 , lim R n D n x 2 ) = (0,x 2 ) ^ (x 1 , x 2 ) . 

n-*°° n-*«> n-*“°° 



Write : 

(3.5) PA r (D) = {xGD oo :lim R n FD n x=0 } 

n ->oo 

where F is an initial operator for DER(X) corresponding to an 
RE^ d , and 

(3.6) PA(D)= rg U^ PA r (D). 

Any element of PA(D) is said to be D-paraanaly tic . Clearly, 
we have 

(3.7) A r (D), Q r , A r (D)©Q r cPA r (D)cPA(D)cD o . 

Thus any D-analytic element is D-paraanalytic , but not conver- 
sely. Also any singular element and a sum of singular and D-analytic 
elements are D-paraanalytic . 

THEOREM 3.3 . Suppose that DER(X) and F is an initial operator 
for D corresponding to a continuous right inverse R. Then PA R (D) 
is a D-R invariant subspace. 

Proof. Write: Y=PA T5 (D) and let yEY be arbitrarily fixed. Then 
R 

lim R n FD n (Dy)=DRlim R n FD n+1 y=Dli m R n+1 FD n+1 y=0; 
n+°o n->°° n->°° 
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lim R n FD n (Ry)=lim R n FD n-1 y=Rlim R n “ 1 FD n-1 y=0. 
n n. 

Thus DyEY, RyEY. The arbitrariness of yEY implies DY cY, RY cY. 

Hence Y is D-R invariant. 

THEOREM 3.4 . Suppose that X is a Banach space, DER(X) and F 
is an initial operator for D corresponding to a quasinilpotent 
RE . Then 

(3.8) H d [PA r (D) ] = PA r (D) 

i.e. the set PA R (D) is D-closed. 

Proof. Formula (3.7) and Theorem 2.3 together imply that H D (0) 
c A R (D)=H D (A R (D) ) cH^PA^D)). 

In order to prove our theorem it is enough to show that yeY=PA p (D) 

-1 K 
implies RyEY, (I-\R) yEY. 

Indeed, by Theorem 3.3, the space Y is D-R invariant. Hence RyEY. 

]r 

By an easy induction we find R yEY for all kElN . Since, by our as- 
sumption, R is quasinilpotent, we have 

(I-\R) -1 = £ k “ 0 \ k R k for all \e<n. 

Hence u=(I-\R) ^GY. 

EXAMPLE 3.2 . Let DER(X) and let and be initial operators 

for D corresponding to right inverses R 1 and R 2 , respectively. Let 
Y=X 1 ©X 2 , where X ± =A R (D) © Q R (i=1,2). Define: 

D(x 1 ,x 2 ) = (Dx^ ,Dx 2 ) for x^XjGdom D? 

R(x 1 ,x 2 )=(R 1 x 1 ,H 2 x 2 ) ; F(x 1 ,x 2 )=(F 1 x 1 ,F 2 x 2 ) for ,x 2 GX. 




252 



It is easy to verify that DER (Y) and F is an initial operator 
for D corresponding to RE . Moreover, if x^=u.j+v^, u i EA R ' 
v.£Q_ for i=1 , 2 , then 

i R i 

lira R n FD n (x 1 ,x 2 )=(lim R^F., D n (u 1 +v 1 ) , lira R"F 2 D n (u 2 +v 2 ) ) =0. 
n^°° n+°° n ^°° 

Hence (x^ ,x 2 )EPA R (D) . We also have 

lim R n D n (x.j ,x 2 ) = (lim R^D n (u^ +v 1 ) , lim R^D 11 (u 2 +v 2 ) ) = 

n -».oo n+°° n^°° 

=(v 1' v 2 ) 

and, in general, 0^ ( v^ , v 2 ) ^ (x^ ,x 2 ) . 

Theorem 3.4 and Theorem 1.4 (ii) immediately imply the following 
COROLLARY 3.1 . Let X be a Banach space and let DCR(X) . Write: 
(3.9) PA (D) =I R £q N ( x) pa r ( d ) f 



i.e. PA°(D) is an algebraic sum of PA_.(D) extended over all quasi- 
nilpotent right inverses R. Then 

(3.10) H D [PA°(D) ]=PA°(D) <= PA(D) 

i.e. the set PA°(D) is D-closed. 

In other words: any solution of a linear equation in D with 
scalar coefficients and with right-hand side belonging to PA°(D) 3 
hence D-paraanaly tic , belongs again to ~PA°(D) 3 hence is also D- 
paraanaly tic . 

Proposition 2.4 (Proposition 2.5, respectively) implies 

COROLLARY 3.2 . Suppose that X is a Banach space DER(X), F is 
an initial operator for D corresponding to a quasinilpotent right 
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inverse R, is either the family of D-shifts or the fa- 
mily of R-shifts. + Write R^=R-F^R, F j 1 ==FS h for and 



PA-(D)=I hem PA p (D) 



*h 



i.e. PA~(D) is an algebraic sum of PA (D) for helR • Then 

R h + 



h d [pa"(d) ]=PA"(D) cPA (D) 



i.e. the set PA~(D) is D-closed. 



Indeed, all R^ are Volterra operators, hence quasinilpotent 
(for the field of scalars is (C) . Recall that in Banach spaces R- 
shifts and D-shifts induced by a quasinilpotent right inverse R 
do coincide (cf. the author [3], also [4]). 



EXAMPLE 3.3 . Let X=C[0,1] over (E , D=gp R=/£. Then R h = /o“/o = /o' 

:ro + 

. (n) 



(F,x) ( t) =x (h) for xGX, hGIR, . It means that PA~(D) consists of all 
h + 



functions such that lim 

n-><» 



(t-h) n =0 for an hGIR + 



In particu- 



lar, PA a (D) consists of functions which are either analytic or 
singular at points hE3R + and their sums. 



Observe that the set PA~(D) contains quasianalytic functions in 
the Mandelbroit sense (cf. [2]). 



NOTE ; We have assumed in several statements that a right in- 
verse R under consideration is continuous. It is well-known that 
if DGR(X) , ker D is closed and an RG^ D is continuous, then D is 
closed. The converse statement is not true, as it is shown by the 
following 

EXAMPLE 3.4 . Let X=C[0,1] with norm sup and let D=g|-. Hence D 
is closed and ker D is closed. However, a right inverse R=/^+cp, 
where <p is an unbounded linear functional, is not bounded (hence 
not continuous) . 
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EXAMPLE 3.5 . The function 

, ,. 1/2 

x(t)=y “ e n * cos(nlt) 
u n= 1 

is infinitely differentiable on the interval [0,2 a] and not ana- 
lytic on [0,2 tt] (cf. Mandelbrojt [2]). 

We have for p63N Q 

x ( 2 P)( t ) = (_ 1 )P Xh » i e- (n ! ) 1 / 2 (n ! ) 2 P C o S (n!t); 
x (2p+1) ( t )=(-l)P^ h ” ie “ (n!)1/2 (n!) 2p+1 sin(n!t) . 

Let R=/^, then (Fx) ( t) =x (0) . We have x^ 2p+ ^(0)=0 for all pE]N o . 

1/2 

However x^ 2p ^ (0)=(-1 ) p £ ~ (n!) 2p ^0. Hence x£Q_. 

u n — 1 r 



J. Rogulski proved that 



x (2p) (0) . 2p 

(2p) ! 



as p-*«> for arbitrary small t^O 



(private communication, February, 1987) . 

Since in our case R 2p FD 2p x=^ — rr-ffl 1 t 2p , we conclude that the func- 

00 d ' * • 

tion xeC [0,2 tt] is not g^-paraanalytic . The proof is too complicated 
to be given here. 



EXAMPLE 3.6 . J. Rogulski proved the following theorem: Let Y be 
a Banach space over 1R and let the set I c]R be open and connected. 
Suppose that f:I -*Y is a function of the class C which is analytic 
on I\{a}, where a£E. Suppose that for a t Q ^a we have 

lim £y(t o -a) n f (n) (a) =0. 

n-00 

Let I o =ID(a- 6, a+ 6) , where 6 = |t Q -a | . Then there exist functions 
f.j:I o -**Y and f 2 :I o such that 

(i) f^ is analytic, f 2 is of the class c” and analytic in I Q Ma}. 

Moreover, f^^ta^O for all nOJ Q ? 
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(ii) f (t) =f 1 (t) +f 0 (t) for all tGI 

1 2 . O 

(private communication, February, 1987) . 

Let X=C(I -Y), D=^r, R=/ fc , (Px) (t)=x(a) for xex. Then D€R(X) , 
ar a d . 

R€QN(X) and the Rogulski' s theorem says that every paraanalytic 

function in C(I -*Y) induced by R is a sum of an analytic function 

and a singular function . 

On the other hand, as Rogulski has shown, there exist ^-para- 
analytic functions which cannot be decomposed in this way. Namely, 
there exists a function f:I -+TR, where I=(-1,1) with the properties 

(i) f is of the class c“ and bounded; 

(ii) f is analytic on I 0} = (-1 ,0) U (0, 1 ) ; 

(iii) f(0)=1, f (n) (0) =2 n n! for all ne® ; 

(iv) f cannot be decomposed on I into a sum f=f<j+f 2 , where f^ 
is analytic on I and f 0 is of the class C°° on I with the 
property that there exists an mG3N such that f 2 ; (0)=0 for 
all n^m. 

By Proposition 1.1 a C°°-function f 2 with the property that 
there is an men such that (0)=0 for all n^m is a difference 

of a singular function and a polynomial of degree m-1 (i.e. an 
analytic function) . 

Having already constructed the function f we define: g(t)= 
f(~ arc tan t) . We conclude that gEC (1) , g is analytic outside 

of zero and bounded. Moreover, g cannot be written as a sum: 

oo 

9 == 9'j + 92' where 9-j is analytic on 3R and g 2 is of the class C and 
has the property that there exists mGIN such that g 2 n ^ (0)=0 for 
n^m. This example was also given by Rogulski. The proofs of these 
facts are too complicated to be stated here. 



4 . SYSTEMS WITH SCALARS COEFFICIENTS 



The results of the previous sections can be extended for systems 
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of equations in D with scalar coefficients: 



( 4 -1) D Cj = I k = 1 a jk?k +T1 k'?k eX < k=1 »•••*“>* 

where DER(X) and a^E-^ for j, k- 1 , . . . , in • 

Indeed, the system (4.1) can written in the following form: 



(4.2) (D-A)x=u, yEY, 



where Y is the Cartesian product of m copies of the space X, 

x=(c r ...,? m )» y=(n 1 --Mii B ). ?i ' • * • ' C m ' ni > — • n m GX and for an 

arbitrary TEL(X) we define 

(4.3) Tx=(Tg 1 , . . . ,Tg m ) for ^ , . . . , £ edomT . 

Then DER(Y) . Let A be a linear operator induced by the matrix 

(4.4) A=(a^ k ) j , k= 1 , . . . , m . 

Clearly, A EL (Y) and A is stationary , i.e. DA=AD on dom DcY 
and RA=AR on Y, where RE-Xp. The operator A is also algebraic . 
Denote by P(t) the characteristic polynomial of A: 

(4.5) P(t)=y k ^ 0 p k t k =nj” 1 (t-tj) where p N =1 , r i+--- +r n =N ' 

t_.^t k for j^k. 

By definition, P(A)=0 on Y. Thus Y is a direct sum 

r . 

(4.6) Y=Y^ <J> . . . © Y n , where Yj=ker(A-tjI) (j=1,...,n). 

The decomposition (4.6) determines n disjoint projectors Pj giving 
a partition of unity : 

(4 * 7) lj=i p j =I ' p j p k =6 jk p j 
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Since the operator A is stationary, all spaces Y.j,...,y are 
invariant under A: 

(4.8) AY czYj (j = 1,...,n) 

and the projectors , . . . ,P R are stationary. 

We therefore conclude that Equation (4.2) is equivalent to n 
independent equations, each one with a polynomial in D with scalar 
coefficients: 

~ N-1 

(4.9) P(D)Xj=yj, where Xj=P^x, y^P^yGdomD 

(4.10) Yj=P o yj+I k ^ 1 P k A k _ 1 (D)yj (j=1,...,n) 

(4.11) A k (D)=I j ^ 0 A k_j D j (k61N o ). 

Indeed, by the decomposition (4.6) we find that Equation (4.2) 
is equivalent n indepent equations: 

(4.12) (D-A)Xj=yj, where Xj=P..x, yj=Pjy (j=1,...,n), 

each one considered on the space Y^ satisfying Condition (4.8). By 
an easy induction we find 

D k x.=A k x.+A, ! (D)y ., y.GdomD^ 1 (j = 1,...,n, k0N). 

3 j k- i 3 3 

Thus we have, for P(A)=0 on Y, 

P(D)Xj=P(A)x j +p o y j +^ k ^ 1 p k A k _ 1 (D) y j =y j (j = 1 , • • • ,n) 

(cf. the author [4]). 

THEOREM 4.1 . Suppose that DER(X) , F is an initial operator for 
D corresponding to an R, A is stationary (i.e. DA=AD on dom D, 
RA=AR) and algebraic with the characteristic polynomial P(t). 
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Suppose that yj ( j=1 , . . . ,n) determined by Formulae (4. 10) -(4. 11) 
belong to D (respectively: to Q_) . Then all solutions of Equa- 
tion (4.2) belong to (respectively: to Q R ©H D (0)). 

Proof. First we have to show that AD^c: D w and AQ R c Q R . Indeed, 
if uED^ then there exists D n u for all nE3N . This implies that 
D n Au=AD n u for all nE3N . Hence AuED . If uEQ- then R n D n u=u for all 

OO K. 

nEIN . Since A is stationary, we find Au=AR n D n u=R n D n Au for all 
nE]N . Thus AuEQ„. If y.ED (respectively: y.EQ 0 ) for j=1 , . . . ,n 
then the j-th equation (4.2) has all solutions belonging to 
(respectively: to Q R ©H D (0)y cf. Theorem 1.5 and 1.4 (vi) , res- 
pectively) . Since x is a solution of Equation (4.2) if and only 
if x=x^+ . . .+x n , where x^ is a solution of the j-th equation (4.9), 
we conclude that x belongs to (respectively: to Q R ©H D (0)) . 

THEOREM 4.2 . Suppose that X is a complete linear metric space 
(over (E) , DER(X) , F is an initial operator for D corresponding 
to a continuous RE V ^ D , A is stationary, continuous and algebraic 
with the characteristic polynomial P(t) . Then 

(i) the spaces A R (D) , A R (D) © Q R , PA R (D) are invariant under A; 

(ii) if X is a Banach space, R is quasinilpotent and y^ (j=1,...,n) 

determined by Formulae ( 4 . 10) - ( 4 . 1 1 ) belong to A„(D) (res- 
ts, 

pectively: to A R (D) © Q r to PA R (D) ) then all solutions to 
Equation (4.2) belong to A^ (D) (respectively: to A_(D) © Q n , 

K K K 

to PA R (D) ) . 

Proof, (i) Suppose that u6A 0 (D) . Then lim R n D n u=0. Since A is 

r n"* 00 

stationary and continuous, we find lim R n p n Au=limAR n D n u=A lim 

n n n - * 00 n - *’ 00 n** -00 

R D u=0. Hence AuEA R (D) . This, and the corresponding part of 

the proof of Theorem 4.1, together imply that the set A_(D) © Q_ 

R 

is invariant under A. Suppose that uEPA p (D) . Then limR n FD n u=0. 

Hence lim R FD n Au=lim A R n FD n u=A lim R n FD n u=0 , what means that 
AuEPA r (D) . The pro8f°°of Point (ii^ is going on the same lines as 
the proof of Theorem 4.1, if we apply Theorems 2.3, 3.1, 3.4, 
respectively. 
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Observe that without any essential change of the proof of 
Theorem 4.2 (ii) we obtain 

THEOREM 4.3 . Suppose that X is a Banach space (over (C) , D€R(X) , 
F j are initial operators for D corresponding to quasinilpotent 
right inverses RjG$ D , A is stationary , continuous and algebraic 
with the characteristic polynomial P(t). Suppose that for 
j=1 , . . . ,n 

E j=D oo or A r (D) or A r (D) © Q r or PA R (D) . 

if y j ( j=1 r - - • ,n) determined by Formulae (4. 10) -(4. 11) belong to 

Ej then all solutions of Equation (4.2) belong to the set E=E,j + ... 

+E . 
n 



In a similar manner we can obtain solutions of Equation (4.2) 
belonging either to PA°(D) or PA~(D) (cf. Corollary 3.1 and 3.2/ 
respectively) . 
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PRODUCTION-POLLUTION CYCLES 
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ABSTRACT. The paper introduces the concept of a firm's "normal" pollution level as a 
weighted average of past "actual" pollution levels. The impact of an incentive scheme 
is analyzed, in which the representative firm pays a penalty (or receives a subsidy) 
when it deviates above (or below) its normal pollution level. The conclusions is, a 
cyclical production and pollution pattern may be optimal under certain conditions. 

1. INTRODUCTION 



There are two broad classes of instruments for pollution control. The first class is 
governmental regulation. It includes prohibition of certain uses, prescribing pro- 
tective measures (e.g. installing smog-suppressors on cars) or setting limits to the 
amounts of effluent factories can discharge into a stream (environmental standards). 
The second class is economic or financial inducement: imposing taxes or changes, or 
offering subsidies to discourage or encourage different ways of using the resource, 
leaving the users free to decide how best to respond to the inducement. 

The present paper deals with the second class of instruments. We consider a represen- 
tative firm which has a certain amount of emission "rights". The firm can use these 
emission rights itself, or sell them to other firms (emission trading), i.e. there is 
a market of emission certificates^. 

The government or the pollution control authority fixes both the selling and purchase 
price of certificates. The main purpose of the following study is to analyze the im- 
pact of an incentive scheme in which a firm receives a reward (or pays a penalty) 
when it deviates below (above) a "normal" pollution level. The result is that such 
an institutional setting may imply a cyclical production-pollution trade-off. 



^ As example we can quote the so-called "bubble-concept" in the air-pollution con- 
trol in the USA (Siebert, 1982). For a general discussion of environmental con- 
trol we refer to Wicke (1982), Baumol and Oates (1975). 
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The paper is organized as follows. Extending a model of Forster (1977) in Section 2a 
two state variable pollution control model is formulated.. In Section 3 the necessary 
optimality conditions of a maximum principle are stated, and the resulting canonical 
system is analyzed numerically. In Section 4 the solutions are presented and inter- 
preted economically. Finally, in Section 5 some conclusions are drawn and possible 
extensions are discussed. 

2. A TWO STATE VARIABLE POLLUTION CONTROL MODEL 

Forster (1977) considers a one state variable pollution control problem. Focusing 
solely on the consumption-pollution trade-off, he assumes that a fixed output is pro- 
duced in each time period. This output is to be allocated to consumption or to pollu- 
tion control (abatement). The society derives utility from consumption but in con- 
suming output, a stock of pollution is accumulated which generates disutility. A cen- 
tral planner intends to maximize the discounted utility/disutility stream over an in- 
finite planning horizon. Using standard optimal control theory, Forster (1977) pre- 
sents conditions under which it is either optimal to eliminate pollution or it is 

not. For both cases he carries out a phase portrait analysis in the state-costate 
phase plane, which yields interesting insight into the transient solution, depending 
on the initial pollution level. 

Feichtinger and Luptacik (1987) investigate a pollution control problem for a profit 
maximizing firm, from the point of view of employment and environmental policy. They 
define an "actual" emission rate, E, which is influenced both by the production and 
the abatement activities. Moreover, they consider also a "normal" (standard) emission 
level, E, assumed to be constant, which is compared with E. If E exceeds E, penalty 
costs are charged to the firm, while for E smaller than E, the representatti ve firm 
obtains a financial reward. 

The present paper combines both approaches sketched above in a microeconomic frame- 
work. The following extensions are made: First, the emission rate E is replaced by a 
stock of pollution P. Second, rather than to consider a constant exogenously given, 
"normal" pollution level P, the environmental standard (normal pollution level) is 
defined as an exponentially weighted average of past pollution levels as follows: 

R(t) = m_^e' m ^ t_s) P(s)ds. (1) 

The parameter m > 0 represents the relative weight given to more recent pollution 
levels. By differentiating (1) with respect to time, t yields ' 



^ A similar adjustment dynamics is considered in Schmalensee (1972), Ryder and Heal 
(1973), Long and Siebert (1985), Steindl et al . (1986), and Luhmer et al. (1987). 
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ft = m(P-R), (2) 

where here and in the following, the time arguments are omitted. For a given initial 
normal level of pollution, 

R(0) = R q , (3) 

and an actual pollution path P(-) the differential equation (2) defines an adjust- 
ment mechanism in the sense that 

ft = 0 for P = R. 

Moreover, both pollution levels have identical stationary values. 

The actual level of pollution, , is accumulated according to 

ft = Z(u) - 6P # P(0) = P Q . ( 4 ) 

Here 6 is the natural cleaning rate, and u denotes the rate of production of the re- 
presentative firm. There is a maximal production rate u such that 

0 ^ u ^ u. (5) 

Z(u) determines the contribution of pollution generated by the production rate u. It 
seems reasonable to assume that Z(u) is convex with 

Z ' ( u ) > 0, Z" (u) > 0. (6a) 

Moreover, there exists a pollution-neutral level of production, u Q e(0,u) such that 

Z(u) = 0 for u = u Q . (6b) 

Here it is assumed that low levels of production make it possible to carry out abate- 
ment activities, by using partly idle production capacities. Similar to Forster 
(1977), the firm's utility is influenced in a two-fold manner by the production rate. 
First, a higher production level clearly generates more pollution which yields dis- 
utility now and in the future. Moreover, each amount of capacity used for production 
can not be used to clean up pollution. 

The level of pollution which is accumulated by Z(u) is assumed to be non-negative: 

P ^ 0. (7) 

The utility rate of the firm is given by 

U(u ,P,R) = it ( u) - D(P) - f(P-R), (8) 



It might appear strange to take Into consideration a pollution level for an indi- 
vidual firm. However, P could be the current amount of slag in a foundry or the 
Mo 77 i 0nS 3 1nto a which is used only by this firm; see, e.g., Sethi 
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where it(u) denotes the net profit of the firm producing at rate u, D(P) is the dis- 
utility induced by the pollution level P, and f(P-R) is a penal ty/ reward function de- 
pending on the excess of actual and normal pollution level. The following assumptions 



are reasonable 

ir(u) > 0, tt ‘ (u ) 


> 0, ir"(u) < 0 


for 


u > 0 


(9) 


D(P) > 0, D'(P) 


> 0, D" (P) > 0 


for 


P > 0. 


(10) 



In the sequel we restrict ourselves to linear penalty functions where the marginal 
penalty (purchase price of certificates) is lower than the marginal reward (selling 
price): 

fa(P-R) for P > R 

f(P-R) = ^ e (p-R) for P < R 

with 

0 < a < B (12) 

Thus, if the actual pollution exceeds the normal level, the firm is penalized in- 
creasingly in accordance to the difference between both levels. On the other hand, 
the firm is rewarded proportionally to R-P provided this difference is positive. 
According to the incentive scheme (11), (12) set e.g. by the government, a market for 
certificates arises. Especially firms which are so small that devices for environ- 
mental protection (e.g. filters) can not be installed, will still profitably pollute, 
and purchase emission certificates to cover the excess P-R. For P < R, the firm can 
sell some of its certificates. Assumption (12) means that the planning authority sets 
a higher selling price to induce environmental protection. 

Thus, the representative utility-maximizing firm is faced with the following infini- 
te-time autonomous optimal control problem: 



max t°e pt U(u,P,R)dt 
u 0 


(13a) 


ft = Z(u) - 6P, P(0) = P Q 


(13b) 


ft = m(P-R) , R(0) = R q 


(13c) 


0 < u ^ u 


(13d) 


P > 0. 


(13e) 



There are two state variables, P and R, and one control, u. Both the initial actual 

pollution level and the normal pollution level are given. The path constraints (13de) 

t) 

are assumed to be automatically satisfied . 



^ For tt( 0) = °°, D'(0) = 0, and u large enough no problems with the path restrictions 
arise. 
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3. ANALYSIS OF THE MODEL 

Since the objective functional is not differentiable for P = R, (13) is not a standard 
optimal control problem. For its solution, a generalized maximum principle has to be 
used (see Clarke, 1983, Hartl and Sethi, 1984, Feichtinger and Hartl , 1985, 1986). 



The current- value Hamiltonian is given by 

H = 7T ( u ) - D(P) - f(P-R) + A[Z(u) - 6P] + pm(P-R) , 
where A and p are the adjoint variables corresponding to P and R, respectively. 

The necessary optimality conditions are 

u = arg max H, i.e. A = -tt 1 (u )/Z ' (u) (15) 

u 

(A-pA,jj-pM)e-a( PjR )H = (D‘ (P)+6A-pm,pm) + 9 ( P#R ) f ( p “ R ) » 
where 



3 (P,R) f ^ P ' R ^ 



{( 6 ,- 6 )} [< 

{(y,-y)|a ^ y ^ 3) for P = R. 
{ (a ,-a )} . >. 



(17) 



Note that the right-hand side of the adjoint equation (16) is discontinuous for 
P = R. a^p p)f( p ‘ R ) is the supergradient of f(P-R) with respect to (P,R) (see, e.g., 
Clarke, 1983). 

To solve the four-dimensional canonical system of differential equations we specify 
the functions U^, and Z as 

ir(u) = Vu", D(P) = P 2 , Z(u) = u 2 - a). (18) 

Moreover, the following values for the parameter values are chosen: 

p = 0.1, a) = 0.1, 6 = 0.1, m = 0.5, a = 0.5, 3 = 1.0. (19) 

If we choose u large enough, e.g. u = 0.5, then restriction (5) never becomes active. 
Thus, the maximizing condition (15) yields 
u = (-4X)- 2 / 3 . 

Hence, the canonical system becomes 
p = (-4a)' 4/3 - (0 - 0.16 ) 



ft = 0.5(P-R) 

x = 0.2x + 2P - 0.5p + y 

(i = 0. 6jj - y. 



( 20 ) 
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where y = 



for 




In order to solve the canonical system (20), the boundary value problem solver COLSYS 
was applied. 



References for the collocation procedure COLSYS are Ascher et al . (1978), Steindl 
(1981); see also Steindl et al . (1986). By choosing appropriate cyclical boundary 
conditions, it turned out that a closed orbit exists. In the following section, we 
discuss the projections of this limit cycle into the (P,R) and (P,u) planes. Further- 
more, the pollution and consumption trajectories are interpreted. 



4. OPTIMAL OSCILLATIONS IN CONSUMPTION AND POLLUTION 
Fig. 1 shows the limit cycle in the state space. 

The cycle is followed counter-clockwise. In the points A 1 and A 3 it holds that ft = 0, 
whereas in A^ and A^ we have ft = 0. In A^ the consumption rate reaches its maximum, 
whereas in A^ it is minimal. 

It seems convenient to distinguish four regimes in the cycle: 

Regime 1 : P > R, ft > 0, ft > 0, Ci < 0. Starting in A^ with low pollution and P = R it 

is optimal to produce at high (maximal) but decreasing intensity. Due to 

the high production level, both the actual and the normal pollution in- 
creases, and penalty costs a(P-R) are charged. This phase ends with maximal 
actual pollution. 

Regime 2 : P > R, ft > 0, ft < 0, Ci < 0. Since the actual pollution is at a high level, 

the production still has to decrease. Due to the low production rate the 

actual pollution decreases. The normal pollution, however, still increases. 
The regime ends as soon as P = R. Here the production intensity becomes mi- 
nimal . 

Regime 3 : P < R, ft < 0, ft < 0, Ci > 0. Since the firm produces at low level, both 

stocks of pollution decrease. It is optimal to increase the production rate. 
A bonus 3 ( R- P ) is obtained. This phase ends with minimal actual pollution. 

Regime 4 : P < R, ft < 0, ft > 0, u > 0. Since the actual pollution is minimal, it is 
still optimal to increase. As result, P increases again. However, R dec- 
reases util the end of this period. As soon as P = R this regime ends and 
the cycle starts again with regime 1. 

The corresponding limit cycle in the (P,C)-phase plane is sketched in Fig. 2. It is 
followed clockwise. 

Figs. 3, 4 and 5 show the corresponding time paths for consumption, actual and normal 
pollution, respectively. 
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Some general insights can be obtained from these diagrams. 

First, it is true that the normal stock of pollution always lags behind the actual 
level (see also (1)). Second, and more important, R(t) and u(t) show opposite be- 
haviour (see Figs. 3 and 5). Third, Fig. 3 reveals that there are essentially two 
production levels, a high rate and a low one: The first prevails during regime 1, 
whereas in regime 3 the firm produces at a low rate. Both regimes 2 and 4 are switch- 
ing phases, which are of short duration compared with regimes 1 and 3. Finally, the 
firm's optimal policy under the given incentive scheme is to produce at a high rate 
if the pollution is low. The increase in pollution due to this behaviour should be 
counteracted by a reduced production. 

A two-dimensional manifold exists in the (P,R,x,p) space containing those paths that 
converge towards the limit cycle. In Figs. 1 and 2, five trajectories converging to 
the limit cycle are depicted. Apparently the convergence is very fast. After one or 
two periods, the paths almost coincide with the closed orbit. 

5. CONCLUDING REMARKS 

There are two main cinclusions. First, we have seen that there is an incentive scheme 
which is responsible for business cycles. Thus, the effect of a certain institutional 
arrangement which aims at the protection of the environment may run counter to its 
good intention^. A useful incentive scheme should not generate cycles of the pollu- 
tion level, instead it should be used to clean up the environment at a reasonably 
economic level. 

Secondly, and more important than this economic conclusion, we have given a numerical 
proof of the existence and stability of a limit cycle in a pollution control model. 
There are a few papers in economic literature where the Hopf bifurcation technique is 
applied to prove the existence of limit cycles analytically (see Benhabib and 
Nishimura, 1979, Medio, 1986, Steindl et al., 1986, Feichtinger and Sorger, 1986). 
However, a complete analysis of this kind is very cumbersome. Not only that Hopf's 
approach gives no general information on the shape of the cycle. This justifies in 
our opinion the use of a numerical analysis. Furthermore, we obtained insight into 
the interdependencies of the model variables responsible for the oscillatory beha- 
viour. A reasonable economic interpretation of the four regimes has been given. 

Following Forster (1977), we have restricted ourselves to the investigation of the 
pure consumption-pollution trade-off. Clearly, more realistic models of environment 
control should include capital accumulation (see, e.g., Luptacik and Schubert, 1982 
ab). Further research effort will concentrate on cyclical investment strategies in 
productive and abatement capital. 



1 ) 



This behaviour is similar, e.g 



to a result of Long and Siebert (1985). 
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LOCATION PROBLEM AND ITS APPLICATIONS 



IN DISTRIBUTION OF PETROL PRODUCTS 



B .Mazbic-Kulma,E .Komorowska, J.StQpieii 



A strategic issue which concerns investment planners 
is where to best locate new warehouses , factories or in- 
termediate stocking points. In this paper we consider the 
location problem of bulk warehouses for petrol products 
in a region. The problem is formulated as a capacitated 
facility location problem. 

Computational results for a set of real-size problems 
are finally discussed. 



1. INTRODUCTION 



The problem of locating warehouses is not new to operations 
research and is known from a rich literature. In this paper 
we consider the location problem of bulk warehouses for 
petrol products in a region. It may happen that transport 
costs are compared to investment costs. Therefore it is very 
interesting where to locate new warehouses in order to mini- 
mize total investment and transport costs. This problem may 
be considered in the framework of designing the efficient 
distribution network configurations. The Desing Office of 
the Petrol Industry was interested in the application of 
this problem and its solutions to a distribution network for 
petrol products. 

Location models can be clossified according to: 




e».p 



278 



1. Whether the underlying distribution network /arcs and /or 
nodes / is capacitated or uncapacitated. 

2. The number of warehous echelons or levels /zero, single 
or multiple/, 

3. The number of commodities /single or multiple/, 

4. The underlying cost structure for arcs and/or nodes 
/linear or nonlinear/, 

5. Whether the planning horizon is static or dynamic, 

6. Patterns of demand /e.g. deterministic or stochastic, 
influence of location ,etc./, 

7. The ability to accomodate side constraints. 

The problem formulations range form single-commodity 
linear deterministic models to multicommodity non-linear 
stochastic models . 

2 . MATHEMATICAL MODEL 

Our problem of warehouses location for petrol products 
distribution network is as follows: 




Fig. 1 . 

- destinations of petrol products, 

- warehouses of petrol products. 
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We have three kinds of warehouses of petrol products. 
The first group contains existing warehouses which have 
fixed capacity. The second group contains existing ware- 
houses too, but there capacity may be extended. The third 
group contains the feasible location places of new ware- 
houses which will be build in future. 

Our problem may be formulated as follows: 



m n nj n 



Minimize 


l 1 (t..+s.)x..+ E k. sgn( I x . . ) 

i=l j=l 13 1 13 i=l 1 j=l 13 


(1) 




m 




subject to 


I x. . = b. ( j=l, 2, . . . ,n) 

i=l 13 3 


(2) 




n 






l x . 4 a. (i=l,2,...,m) 

j=l 13 x 


(3) 




x i; j * 0 (i=l,2, ,m) , ( j=l, 2, . . . ,n) 


(4) 



Where: 



X ij " 



*i' 



=1 



C ij " 



b. - 
3 

a i - 
m - 



the number of units of customer j's demand satisfied 
by facility i. 

if facility i is established, 0 otherwise 

the unit investment and distribution costs associated 

with satisfying customer j's demand from facility i 

(c ±j = hj + s i ) 
the fixed costs of facility i 

the demand of customer j 

the capacity of facility i 

the number of potential warehouse's location 



n 



the number of customers. 
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This problem may be solved by the method of succesive 
calculations | 7 | , | 24 | . 

Let M = {l,2,...,m} denote the number of possible 
locations of warehouses. Let ft , where u € ft , denote 
the set of all subsets of the set M. A subset w will be 
called a variant of locations. 

Of course the number of all subsets of the set M is 
2 m , i.e. the maximal number of possible solutions is 2 m 
too . 

Let F be a function defined on the set ft . In order 
to reduce the number of combinations of plant location 
to study, we can use the following property and theorem. 

Cherenin Property 

For arbitrary o^, w 2 ^ 

F ( a) 1 ) + F ( (jq 2 ) - F(o) 1 U a> 2 ) - F^Au)^ 0 (5) 

Cherenin Theorem 



Let a function F(io) possess Cherenin Property and 

let a) be an optimal subset. Then, for any finite 

sequence {to.} containing w* , where C ,the 

is. 

function F(w) is monotone nonincreasing to oj and 

it 

monotone nondecreasing after . 

The function wich posses this theorem is called sub- 
modular. It is proved by Babayev that Cherenin Theorem 
and properties formulated by Frieze for submodular fun- 
ctions are equivalent. 
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The quality of a location variant may be evaluated by 
the following function F(oj) defined on the set n. 



F(u) = 


E 


k i + 


L( 03 ) 




(6) 




ie a) 


n 








L( to ) = 


min 


E 

ie a) 


E c . . X . . 
j=l ^ ^ 




(7) 


subject 


to 


E 

ie a) 


x . . > b . 
ID D 


( j 1/2, .. . ,n) 


(8) 






n 












E 

j = l 


x. a . 

ID 1 


i e o) 


(9) 






X . . 

ID 


* 0 , i £ o) 


f (3~l/2,...,n) 


(10 



It may be proved, that this function possess Cherenin 
Property . 

F(u)^)+ F ( co 2 ) ~ F(o) 1 U u^) " F(w 1 n 03 2 ) £ 0 




Fig. 2 . 




